Anybody seen anything like this? It came from
66.207.121.61 and it did not fetch robots.txt so it grabbed all html files including disallowed ones. It also switched User-agents four times within two minutes:
- Mozilla/4.0 (compatible; MSIE 5.01; Windows 98; P-P-C Cancel)
- Mozilla/4.0 (compatible; MSIE 5.01; Windows 98; {World Online})
- Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)
- Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0; pide-09-00)