Forum Moderators: open

Message Too Old, No Replies

funny request for robots.txt

error or on purpose?

         

the_nerd

8:28 am on Jul 3, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



slurp left this in my logfile:

http:/www.mydomain.com/robots.txt

Any Idea why they would want to get my robots.txt with a wrong protocol?

nerd

engine

5:51 pm on Jul 4, 2006 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



Was it for sure genuine article?

Pfui

6:11 pm on Jul 4, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Slurp's been a bit sloppy lately --

Naughty Yahoo User Agents
[webmasterworld.com...]

-- but I've never seen it use a domain in a request. Echoing engine, more info would help, like the full UA, and the Host/IP, etc.

(The linked thread includes lots of UAs and such, including Slurp.)