Page is a not externally linkable
waynet - 8:12 pm on Nov 25, 2005 (gmt 0)
I was also thinking one of those Google search appliances would be a good replacement for the site search, but looking at it and the number of pages here it would cost a fortune. Brett, know anybody over there that would give you a good discount on that? Maybe as part of some beta test or something? [edited by: Brett_Tabke at 9:51 pm (utc) on Nov. 27, 2005]
Anyway, I was thinking if it would be possible to set up a mirror site where posting wasn't allowed and redirecting all known and detected robots over there, would that work? Let the bots chew on a slow mirrored site and keep the real site relatively bot free. When a user logs into the mirror site, have them redirected over to the real site. At least Google will have something to crawl.