Welcome to WebmasterWorld Guest from 22.214.171.124
Forum Moderators: phranque
Is there a technical way to respond to the spider: "Hey, I'm busy right now; please come back later, and by the way, don't penalize me for responding in this way". (Note that the discussion board provides for me to determine the current user load that I would use in determining when to respond in this manner.)
Hits to a discussion board suck up database resources, even with SQL caching built in, so I need to be able to tell the spiders to not bother the site for a while.