Welcome to WebmasterWorld Guest from 34.201.121.213

Forum Moderators: Ocean10000

Message Too Old, No Replies

New Yahoo crawler?

Strange crawler from yahoo

     
6:21 pm on Mar 11, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 19, 2003
posts:136
votes: 0


Had an odd crawler hit one of my sites that is coming back with a Yahoo ip address. It identified as crw3.vc.search.re2.yahoo.com. Anyone know what this is for?
8:35 pm on Mar 11, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 22, 2001
posts:2450
votes: 0


What's the User Agent?
2:51 pm on Mar 15, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 19, 2003
posts:136
votes: 0


The user agent came back as crw3.vc.search.re2.yahoo.com.
4:03 pm on Mar 15, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2001
posts:5506
votes: 5


"crw3.vc.search.re2.yahoo.com"

These items, what ever they are referred to have not been used by any of the hosts I've had. I do realize that some hosts do provide these references (what ever their called) rather than IP ranges.

george's canufly provided the Yahoo range:
68.142.230.180

User Agents are the last field in a full log.
Some examples here;
[pgts.com.au...] (large file and slow loading)
[clearwaterbeachcam.com...]

A typical Yahoo UA is:
"Mozilla/5.0 (compatible; Yahoo! Slurp; [help.yahoo.com...]

5:54 pm on Mar 16, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 19, 2003
posts:136
votes: 0


Wilderness, thanks for the reply. I am aware of what a user agent is and how they normally show up. I have double checked the logs and crw3.vc.search.re2.yahoo.com is what is appearing as the user agent.

My question is if anyone knows the purpose of this crawler and if it is really associated with yahoo.