Forum Moderators: open

Message Too Old, No Replies

Altavista crawling heavily

Why does AV stick around so much?

         

amoore

5:52 am on Mar 10, 2002 (gmt 0)

10+ Year Member



I've noticed that Scooter-W3.1.2 from Altavista crawls one portion of my site a lot, to the tune of once every two hours, and has been for a few weeks. It's only a few dozen pages, but when you crawl them that much, it adds up. Scooter is constantly one of the useragents that pulls the most bandwith from me.

Has anyone else experienced this kind of behavoir or similar before? Or can you venture a guess why it's doing that?

For what it's worth, it's not a very unusual site, just some original content on what look like pretty much static pages. They're new within the past few months, though. I can't find any of them in altavista's engine either.

Any reason that I shouldn't just deny it access to that area?

Brett_Tabke

6:55 am on Mar 11, 2002 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



I've not seen that type of specific behavior before. If you aren't getting any referrals from AV, then shut them off.