Forum Moderators: open

Message Too Old, No Replies

Yahoo Excessive Crawling.

crawling 500+ pages over and over again yet NO visitors

         

rajannewton

12:01 pm on Jul 20, 2010 (gmt 0)

10+ Year Member



A while ago I noticed that on one of my sites, crawling 500+ pages over and over again yet NO visitors. i cheked and its yahoo crawler so I blocked Y from crawling, they're wasting my bandwidth. Each month 35% of all bandwidth is used by Yahoo Slurp. Pl. suggest what we do to slow a yahoo crawler.

pathmaker

3:30 am on Jul 22, 2010 (gmt 0)

10+ Year Member



I experience that same thing too. please do suggest ways to slow down yahoo crawler.

zeus

10:39 pm on Jul 31, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



yep same here was like ddos attack server went totally slow

skweb

3:15 pm on Aug 2, 2010 (gmt 0)

10+ Year Member



Ranking and crawling are two entirely different things. Unfortunately if you want to be found you have to pay for bandwidth. So blocking a search engine is not the answer. The key is to build links so that you can also raise your website's rankings.