Welcome to WebmasterWorld Guest from 54.162.166.214

Forum Moderators: werty

Message Too Old, No Replies

Watch out for crawler.searchmarketing.yahoo.com

Badly behaving bot

     
6:51 pm on Apr 1, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Sept 27, 2003
posts:156
votes: 0


Just a word of caution to anyone using Overture. Watch out for the bot, crawler.searchmarketing.yahoo.com (IP 66.35.192.197).

This guy is currently behaving really badly, sending an exorbitant amount of simultaneous requests sustained over long periods of time. It brought our server down on its knees several times. Essentially it was acting as a Denial of Service attack.

We've had to temporarily ban the bot until Yahoo can tame this wild beast and make it be a bit more considerate [hey Yahoo, not everyone has a server farm you know].

8:30 pm on Apr 1, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:May 24, 2002
posts:894
votes: 0


I'm inclined to believe that this is a spoofed Y! bot.
The IP number does not belong to Y!
8:47 pm on Apr 1, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Sept 27, 2003
posts:156
votes: 0


well then it's a damn good spoofer because it only hits sites which are running overture ads, and it only hits the ad urls. And increased # of ads coincides with increased activity from this bot. I am told it is their automated ad checker.
8:58 pm on Apr 1, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Sept 27, 2003
posts:156
votes: 0


And also, searchmarketing.yahoo.com resolves to 64.209.232.29. Yahoo is not listed as the owner of this IP either, but Savvis is, the same as the IP of crawler.searchmarketing.yahoo.com
5:04 pm on Apr 4, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 20, 2004
posts:1477
votes: 0


It's the automated ad-checker ... but nobody at Y will acknowledge its existence. We have gone through a few extended battles with those bots over the past couple of years, and Y never admitted what was happening. The DoS aspect of the probing would certainly make for an interesting lawsuit ...

We started blocking the bots' IPs, and discovered that doing so kept us from receiving mail from SBC-Yahoo subscribers ... leading us to believe that not only are there unattended bots running from those servers, but that they are sharing the space with email users. When we unblock an IP based on an email message's headers so we can receive mail from that subscriber, the bots return.

10:28 pm on Apr 13, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Apr 12, 2004
posts:63
votes: 0


I checked my logs and it is the only IP address that shows up in my reports for my yahoo ppc campaign. Should I report this to yahoo or is this a valid ip address?
2:37 pm on Apr 14, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Aug 14, 2005
posts:64
votes: 0


I have seen this too. Every time we do a bulk upload, the bot brings our server down. Way to be professional, YSM!
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members