About/0.1libwww-perl/5.47

[Marcia]

209.143.212.23 - - [22/Sep/2001:05:32:20 -0400] "HEAD / HTTP/1.0" 200 0 ""-" "About/0.1libwww-perl/5.47

I'm wondering what this is looking for. This site got a link from about.com and gets some referrals. I haven't seen this with any of the others. Is this just to make sure the link is still active?

[littleman]

The IP is part of a class C which belongs to [globalcrossing.com...] . They could be the parent company for your ISP? The "libwww-perl/5.47" of the UA is a giveaway that the bot is probably a simple script written in perl. It made a HEAD request to your root page, so it could be checking on the status of your server/website. I don't see any links between Global Crossing and about.com

[littleman]

Scratch what I said about the connection with about.com....

[littleman]

209.143.212.20 -> ny-www.about.com
209.143.212.21 -> apps1.about.com
209.143.212.22 -> nyimages.about.com
Lookup for 209.143.212.23 failed.
209.143.212.24 -> insideprimedia.about.com
209.143.212.25 -> apps.about.com
Lookup for 209.143.212.26 failed.
Lookup for 209.143.212.27 failed.
209.143.212.28 -> primesearch.about.com

As you could see the IP sits in the middle of several of About's domains. So your hypothesis is likely. So, about uses a little perl script to do link verification... :)

Just goes to show you how poking around in neighboring IPs could pay off.