Forum Moderators: open
12.10.127.130 - - [03/Sep/2003:08:21:33 -0700] "GET /robots.txt HTTP/1.0" 200 535 "-" "Mozilla/4.0 (compatible; MSIE 5.0; Windows NT)".
It's grabbing pages every 1-4 seconds. It hasn't gone to any unauthorized places yet, however, I don't trust this sucker.
Anyone else seen this bugger or know anything about it?
I've checked their website and can find no information regarding a bot/spider.
I've sent an email to them asking why they feel a need to spider a site that has nothing to do with MO. or travel.
Meanwhile, I've banned them. 2 weeks of this is enough.
and exhibits the same crawling pattern.