Hope - 1:26 pm on Oct 16, 2000 (gmt 0)

This may sound like a stupid question, but I am gonna ask anyway. I was always told the only stupid questions are the ones you don't ask.

How do I slow slurp down? Our entire website is spidered every week. That is over 5000 page views a week by slurp for the last year or so. We do update our site regularly, but not the entire site every week. I really don't see a need to have the entire site spidered that often.

I know that I can add slurp to the robots.txt, but that would block it. That is not what I want. I just want it to come once a month and spider the whole site, or something like that.

Any suggestions would be greatly appreciated.