Last month, this Host/IP behaved itself after requesting robots.txt where it's Disallowed. This month? equested, and ignored:
134-163-253-248.shell.com [
projecthoneypot.org...]
Goldfire Server
02:34:38 /robots.txt
02:34:57 /
134.163.253.248 is a.k.a. irongrove.net [
robtex.com...]
"Goldfire Server" is not a completely uncommon UA (although I reckon it's wrongly configured when it's the UA). So why is shell.com spidering, I wonder? SERPs from 2000-2005 suggest it was an employee-accessed server, or some such. [
g6ftpserver.com...] Now? Dunno. Anyone else seeing this?