Is internap crawling for ink? They are exhibiting crawling behavior: 220.127.116.11 18.104.22.168 22.214.171.124 126.96.36.199 188.8.131.52 184.108.40.206 220.127.116.11 18.104.22.168 22.214.171.124 126.96.36.199
They were hitting a ton of my domains for a few weeks and I gave em a list of my IP's so that they would stop. They weren't eating up much bandwidth, just filling up the raw log files.
I "think" for now they are just scouring the web to find the best route to access various parts of the country or world. Like if there is a lag from point A to B to C to D then re-route the connection so that it goes from point A to E to F to D. This would explain them finding sites by IP address instead of DNS.
They might have some other uses for this in the future if they have not already implemented it. I could imagine someone like Inktomi using this to their advantage on their search engine database.
First they rank all the sites they spidered in terms of relevance, then rank them in terms of connectivity. Whoever has the best of both worlds gets a better listing. Or if a site is down too often then it gets dropped out of the index.