Forum Moderators: DixonJones
I see this thing also. It checks robots.txt first, so I ignore it.
But coyote, what makes you think that it is an "official" bot from the edu? Anyone connected through the university's IP range could be running their own personal bot.
For example: I'm WiFi and have a login key to our local Univ IP (ex staff), so anywhere within a 3 mile radius of the school, I can sit under a tree and run a copy of BlahBot from my laptop and crawl your site ;)
sherlock_spider (jimfan@163.com) [google.com].
Check out the cached versions too.
129.79.245.98 (burrowww.cs.indiana.edu) comes up on one.
Smells like that one college a couple months back that had an open relay being run by five folks in that tech department. Least ways all the mails to that department bounced as no longer deliverable and I'd just pulled the data from the college website. Go figure.