Forum Moderators: open
It came through my personal site on Wednesday. Started with robots.txt then requested about a 100 files (including two ending in .pl, and one .html with a "?pref=" pseudo-dynamic trailer). Preceded every GET with a HEAD request, for a total of 213 log entries. (Strangely enough, it still issues GET requests for things that HEAD said were 301.) In and out in 6 minutes, 34 seconds; hasn't returned for the rest of my site (yet). Seemed to obey robots.txt, but I'm not sure, since it also skipped areas that are OK to spider.
I can't find anything about it on activestate.com (but they don't have site search, so I easily could have missed something), and a Google search only turned up a few "server statistics" pages (oldest one from January), and a Webmasterworld search showed nothing.
I've already sent a message to ActiveState asking them what UniverseBot is. They're a software company (I use their version of Perl on my PC), so I'm thinking it might be a test of new product.
Anybody know anything more solid?
ActiveState replied to the message I sent them about it. They said UniverseBot was "checking for mentions" of their company on the web. (I actually do have one link to them, recommending their version of Perl, so I suppose it's possible they found me via referral logs.) They also said they've retired UniverseBot and replaced it with "a Google bot".
I think I'm going to keep an eye on their IP range, to see what "a Google bot" might be.
