Welcome to WebmasterWorld Guest from

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies


1:40 pm on Nov 3, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 5, 2005
posts: 2038
votes: 1

Last month, this Host/IP behaved itself after requesting robots.txt where it's Disallowed. This month? equested, and ignored:

134-163-253-248.shell.com [projecthoneypot.org...]
Goldfire Server

02:34:38 /robots.txt
02:34:57 / is a.k.a. irongrove.net [robtex.com...]

"Goldfire Server" is not a completely uncommon UA (although I reckon it's wrongly configured when it's the UA). So why is shell.com spidering, I wonder? SERPs from 2000-2005 suggest it was an employee-accessed server, or some such. [g6ftpserver.com...] Now? Dunno. Anyone else seeing this?
7:23 pm on Nov 3, 2011 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
votes: 64

Pretty much the same experience, however unsure about the range. I'd see Goldfire Server once every several weeks. Behaved and of no concern. Then last week it ignored robots.txt and crawled a few pages. Now blocked by UA.