Forum Moderators: open

Message Too Old, No Replies

Is this really Yahoo's slurp?

         

botslist

5:04 pm on May 17, 2007 (gmt 0)

10+ Year Member




user-agent: Mozilla/5.0 (compatible; Yahoo! Slurp; [help.yahoo.com...]
via: 1.1 mt501011.inktomisearch.com:3128 (squid/2.6.STABLE12)
x-forwarded-for: 127.0.0.1
x-peer: 74.6.79.142

I just received an email from my bot monitoring site with the above information and its puzzling me a bit. It looks like a client program on localhost (from the x-forwarded-for header) was connecting to a squid proxy on port 3128 of Yahoo's machine (from the via header). The connection to my site is from the host indicated in the x-peer header, and dns matches this host with the one specified in the via header. Anyone seen this before?

Ocean10000

12:56 am on May 22, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Took me a bit of checking but yes I have seen this from Yahoo Slurp before, this last December (12/01/06). I determined at the time it was really Slurp. But why they were where using proxy server I have no idea.

Here is the header data I have for when I was visited, on one of the sites I control.


Cache-Control: max-age=51840000
Connection: keep-alive
Via: 1.0 lj612345.inktomisearch.com:3128 (squid/2.6.STABLE5)
Accept: */*
Accept-Encoding: gzip, x-gzip
User-Agent: Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
X-Forwarded-For: 72.30.177.10

For my Reference later
Test header id=31581