Slurp Activity is Up - 200+ visits a day

[roodle]

Hi,

One of my sites is getting 200-300 visits a day from this robot:

Mozilla/5.0+(compatible;+Yahoo!+Slurp;+http://help.yahoo.com/help/us/ysearch/slurp

even though I get practically no traffic from Yahoo (the site in question ranks badly on Y). WHY is this robot insisting on hammering my site? It is a database-driven site, but that's not a reason I don't think. I don't really want to have to ban the robot, but if this carries on I might have to.

This is my first post on the Yahoo forum so if you've already discussed this type of thing please post the link to the thread.

Thanks.

[guru5571]

no idea, but they have been doing the same to me for months and I rank poorly with them.

[miedmark]

I have been getting about 300-400 daily visits from slurp. I have been ranking poorly with yahoo as well - last week things have improved and I actually get some traffic from yahoo. Maybe the same will happen to you.

[Dijkgraaf]

How do you measure "visits"?
If it is from a stats program, then what they may be counting is how often it requests robots.txt, and Slurp requests this often.

[eyezshine]

Slurp is allways busy. I even have a site that is banned in yahoo search and slurp is allways busy spidering that site every day like nothing has happened?

I have also thought about banning slurp from the "banned" or whatever site to save on bandwidth but I have also seen sites pop in and out of their results from time to time so I don't know what to do?

If the site is truly banned I wish they would tell me so I can ban the bot because it's a pain to be wasting bandwidth for no reason. At least googlebot quits spidering when a site is banned.

[roodle]

I banned slurp and it still comes back on average once an hour to check the robots.txt. Each time it returns is classed as a "visit" and is recorded as such. But before the ban it was visiting around 200+ times a day, and these visits were warping my stats. I needed to lower the bandwidth too, so since the site in question ranks so badly in Yahoo and gets so little traffic from it, I thought banning Slurp was the best option.

[trinorthlighting]

I am getting the same amount of visits from slurp also. Seems like it is going crazy on my site. May be yahoo is getting ready to finally index my site.