Yahoo!-AdCrawler

[GaryK]

Mozilla/5.0 (compatible; Yahoo!-AdCrawler; [help.yahoo.com...]

This one was just brought to my attention by someone. I can't find anything about it here on WebmasterWorld. The URL is valid and contains info about Yahoo's ad crawler. I have no idea if it reads/respects robots.txt.

Has anyone seen this one before?

[Ocean10000]

I found some references to it on the net in some public web logs. Coming from some Yahoo owned space. I have seen some references to it going back to late 2008, but do not have access to those raw logs to say more then that.

2009-09-23 - 74.6.22.182
2009-09-06 - 74.6.18.222

Only thing in common I can see on the websites is they all had adsence on them. They might also show other advertisers as well I think so this following quote from the user-agent link I think is accurate.

Recently, Yahoo! launched another crawler (named “Yahoo!-AdCrawler”) that also will crawl YSM advertiser landing pages. This crawler is designed to help us make relevance assessments of these pages, that helps us match ads to specific search queries.

[GaryK]

they all had adsence on them

Is there some collaboration between Y! and Google that I'm unaware of, Owen?

[Ocean10000]

No. I think the sites in question, either show Googles or Yahoo adverts. I just happened to visit when they were showing Googles in this case.

[GaryK]

Oh, okay. Thanks. :)