bitlybot

AWS again

[GaryK]

bitlybot
75.101.136.nnn
ec2-75-101-136-nnn.compute-1.amazonaws.com
-----
OrgName: Amazon.com, Inc.
OrgID: AMAZO-4
Address: Amazon Web Services, Elastic Compute Cloud, EC2
NetRange: 75.101.128.0 - 75.101.255.255

ROBOTS.TXT? No

Took the default root page for my browser project, the only site anything using AWS can access.

Bitlybot sounded so familiar to me at first. Then I realized I was thinking of the URL shortening service bit.ly. I wonder if there's any connection. I couldn't find anything connecting the two using Google though.

[keyplyr]

Yes, it's the Twitter short URL utility, not really a web crawler so robots.txt may not be applicable. Many of these new Twitter apps use Amazon Web Service which is why I let up my EC2 IP range ban. Twitter generates a lot of traffic for me lately.

[GaryK]

Can you explain to me why bit.ly needs a bot of any kind? Does it go get the URL someone is making a short link to?

[keyplyr]

I believe it archives for its other services.