Forum Moderators: open

Message Too Old, No Replies

New Yahoo crawler?

Strange crawler from yahoo

         

Strange

6:21 pm on Mar 11, 2005 (gmt 0)

10+ Year Member



Had an odd crawler hit one of my sites that is coming back with a Yahoo ip address. It identified as crw3.vc.search.re2.yahoo.com. Anyone know what this is for?

volatilegx

8:35 pm on Mar 11, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



What's the User Agent?

Strange

2:51 pm on Mar 15, 2005 (gmt 0)

10+ Year Member



The user agent came back as crw3.vc.search.re2.yahoo.com.

wilderness

4:03 pm on Mar 15, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



"crw3.vc.search.re2.yahoo.com"

These items, what ever they are referred to have not been used by any of the hosts I've had. I do realize that some hosts do provide these references (what ever their called) rather than IP ranges.

george's canufly provided the Yahoo range:
68.142.230.180

User Agents are the last field in a full log.
Some examples here;
[pgts.com.au...] (large file and slow loading)
[clearwaterbeachcam.com...]

A typical Yahoo UA is:
"Mozilla/5.0 (compatible; Yahoo! Slurp; [help.yahoo.com...]

Strange

5:54 pm on Mar 16, 2005 (gmt 0)

10+ Year Member



Wilderness, thanks for the reply. I am aware of what a user agent is and how they normally show up. I have double checked the logs and crw3.vc.search.re2.yahoo.com is what is appearing as the user agent.

My question is if anyone knows the purpose of this crawler and if it is really associated with yahoo.