Altavista Scooter eats up too much trafic - Sitemaps, Meta Data, and robots.txt forum at WebmasterWorld - WebmasterWorld

Forum Moderators: goodroi

Message Too Old, No Replies

Altavista Scooter eats up too much trafic

maxidrom11

5:28 am on Jan 24, 2004 (gmt 0)

Hi
can somebody help me how to make a robots.txt so that Scooter visits my web site noy so often?

MarkHutch

5:28 am on Jan 24, 2004 (gmt 0)

WebmasterWorld Senior Member

10+ Year Member

You can disallow it, but I don't think you can slow it down. How fast is it getting your pages?