Welcome to WebmasterWorld Guest from

Forum Moderators: werty

Message Too Old, No Replies

Yahoo Product Submit Crawlers Not Behaving?

11:40 am on May 6, 2008 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member 10+ Year Member

joined:June 12, 2003
posts: 702
votes: 9

Anyone out there use Yahoo Product Submit notice that the spiders have two main issues:

(1) The spiders don't seem to be properly identifying themselves
(2) The spiders often crawl a single site at a rate of 20 pages per second (which is incredibly high crawl rate).

Anyone else notice that?