Welcome to WebmasterWorld Guest from 54.145.208.64

Forum Moderators: werty

Message Too Old, No Replies

Yahoo Product Submit Crawlers Not Behaving?

   
11:40 am on May 6, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Anyone out there use Yahoo Product Submit notice that the spiders have two main issues:

(1) The spiders don't seem to be properly identifying themselves
(2) The spiders often crawl a single site at a rate of 20 pages per second (which is incredibly high crawl rate).

Anyone else notice that?