Internap? Crawling for Inktomi?

[Brett_Tabke]

Is internap crawling for ink? They are exhibiting crawling behavior:
12.27.166.57
198.107.213.57
206.229.153.57
206.64.105.57
206.98.113.57
207.86.73.57
208.47.242.57
208.51.235.57
216.223.48.225
4.20.90.57

Agent and referer:
[internap.com...]

Partners with Inktomi:
[internap.com...]

[msgraph]

They were hitting a ton of my domains for a few weeks and I gave em a list of my IP's so that they would stop. They weren't eating up much bandwidth, just filling up the raw log files.

I "think" for now they are just scouring the web to find the best route to access various parts of the country or world. Like if there is a lag from point A to B to C to D then re-route the connection so that it goes from point A to E to F to D. This would explain them finding sites by IP address instead of DNS.

They might have some other uses for this in the future if they have not already implemented it. I could imagine someone like Inktomi using this to their advantage on their search engine database.

First they rank all the sites they spidered in terms of relevance, then rank them in terms of connectivity. Whoever has the best of both worlds gets a better listing. Or if a site is down too often then it gets dropped out of the index.

[Brett_Tabke]

Appears it is for their cache servers thought, and not the se. The se stuff is coming out of Exodus (who I'd love some more info on if anyone has it - who owns them, who started them...etc).

[msgraph]

>>Appears it is for their cache servers

I agree. From reading through Inktomi's Traffic Server info page it looks like InterNap is a real influence on how their server functions.

Here is some info on them financially if you haven't already come across it.

InterNap [cnnfn.cnn.com]

[Mike_Mackin]

JANUS CAPITAL CORP may have started them.

[sec.gov...]