Forum Moderators: open

Message Too Old, No Replies

Does Yahoo Japan have its own Bot

         

Vimes

5:07 am on Nov 8, 2006 (gmt 0)

10+ Year Member



Hi,

Does Yahoo.co.jp have its own Useragent on its robot.

looking at thh other Languages it should arrive like this:

Mozilla/5.0 (compatible; Yahoo! JP Slurp; ht*p://help.yahoo.com/help/us/ysearch/slurp

can anyone confirm thats what they use?

Vimes.

GaryK

11:10 pm on Nov 9, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



So far Yahoo! Japan has mostly sent us sneaky bots with no user agent or misleading user agents, all of which ignore or disrespect robots.txt.

This is the only confirmed user agent I've seen from Yahoo! Japan and it's been quite awhile since I last saw it.

Mozilla/4.0 (compatible; Yahoo Japan; for robot study; kasugiya)

Vimes

4:35 am on Nov 10, 2006 (gmt 0)

10+ Year Member



Thanks Gary much appreicated.

Vimes.

incrediBILL

8:26 pm on Nov 10, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I think half the people working at Yahoo have their own bots, but that's just a theory.

GaryK

5:14 pm on Nov 12, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Interesting find in this week's log files:

Mozilla/4.0 (compatible; Y!J; for robot study)
203.216.197.nnn
*.yahoo.co.jp

EDIT: Of course it did not read robots.txt. ;) It was only interested in one page which happens to be my #1 page in the SERPs for various keywords related to my browser project.

[edited by: GaryK at 5:16 pm (utc) on Nov. 12, 2006]