Forum Moderators: open

Message Too Old, No Replies

Is this Yahoo crawling via Google?

         

GaryK

2:23 am on Dec 14, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp),gzip(gfe) (via docs.google.com/viewer)

74.125.154.nn
No PTR
-----
NetRange: 74.125.0.0 - 74.125.255.255
NetName: GOOGLE
-----
It took a few PDF files and left. The UA is most likely spoofed, right?

wilderness

2:12 pm on Dec 14, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Hey Gary,
How goes it?
There some reference to Yahoo utilizing google docs and related tools, and as related to mobile products.

I'm guessing that Yahoo is just spidering the google docs which their system is using.

dstiles

10:15 pm on Dec 14, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Wilderness, yahoo can't (easily) trawl through a google IP. :)

My guess is that the bot is crawling in IP range .154.80 - .154.89. If so I have that range set to Always Ban with the note: "bad accesses from several IPs in range." They are certainly not googlebot IPs. In fact the whole 74.125/16 looks like a general bot farm that includes web preview, translate and similar.

My giess on Gary's hit would be google pretending to be yahoo for some nefarious purpose of their own OR someone somehow browsing with a fake UA through a google proxy - my money would be on the former. :)

GaryK

1:20 am on Dec 15, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hi Wilderness. I'm doing alright thanks. Hopefully you (and everyone else here) are having a good holiday season.

dstiles, you'd be right about your guess, it's 154.80.

I'd love to ban the entire netrange, but one of my sites has more non-native English speakers than native ones and they depend on Translate a lot.

Thanks, guys. :)

wilderness

3:09 am on Dec 15, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Wilderness, yahoo can't (easily) trawl through a google IP.


dstiles,
Realized that as soon as I hit the submit :(

My real point was that there is some integration between Yahoo and Google Docs.

GaryK

5:24 am on Dec 15, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



My real point was that there is some integration between Yahoo and Google Docs.

Really?