How To Block trek6.sv.av.com from my site

trek6.sv.av.com takes up too much bandwidth

[akogo]

Anyone know how to block trek6.sv.av.com? Do I do this in my robot.txt file? If so, what is the command or code? It has already used up 106294 kilobytes of bandwidth on my site within the last couple of days.

[Slade]

Did it, or does it request robots.txt when it visits? If so, then you can try blocking it that way.

If not, you can block the IP.

For apache, search here for "htaccess ban lists" to find good info.

On IIS, you can block that IP in the Directory Security tab, I think.

[fiestagirl]

If you would like to disallow:

User-agent: scooter
Disallow: /

It's AltaVista. They usually check robots, but have been known to get out of hand.

[edited by: fiestagirl at 4:53 pm (utc) on Aug. 2, 2003]

[akogo]

Thank you Slade and fiestagirl. Have you had problems with spiders that cause too much bandwidth use?

[fiestagirl]

Well I have to say that depends on what you mean by too much bandwidth use. I personally allow scooter and ban things that I really detest like censor bots, trademark bots, plagarism bots. Every webmaster has to make that call for him/herself. From Scooter I get some traffic. The other ilk are selling information about me and not giving me a cut. (:

[jimbeetle]

Welcom akogo,

If you are getting any traffic from Alta Vista, or want to get any traffic from AV or any other search engine, think twice, then stop and think again before blocking them.

In general, while most traffic might come from Google, MSN and Yahoo, the bit of traffic 10 to 20 other engines send your way can turn out to be a substantial number of visitors when figured over a year's time.

used up 106294 kilobytes of bandwidth

How many pages did it suck up? If you have a large site and it spidered a couple or so thousand pages that might be a reasonable amount of bandwidth (or a smallish site with constant changes in content). If it keeps asking for the same pages over and over again then you might have a problem in your site structure or navigation that needs to be addressed.

Either way, you make the call as to which spiders you allow in.

Jim