Forum Moderators: open
The Yahoo spider "crawl1.rides.auto.mud.yahoo.com" has been consuming some pretty big bandwidth on one of my sites lately... Almost 7GB for each of the last two months.
To give you an idea as to the signifigance of this, the next highest spider consumed 270MB this month, and Yahoo traditionally has consumed about 50MB per month.
The site has always had pretty good traffic and rankings with Yahoo but recently the bandwidth surge has me a little confused. This site has about 23,000 pages in the Yahoo database.
Anyone else seeing unusually high spider traffic from Yahoo lately, or have ideas on how to tell this spider to take a little easier on my site? =)
Thanks!
DNS tools provides this IP.
209.191.64.227
Dan keeps this list relatively current and does NOT provide the above range ad Yahoo.
[iplists.com...]
Don