Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Yahoo! Slurp China


jdMorgan - 3:51 am on Dec 1, 2005 (gmt 0)


"Never give up, never surrender..."

Yes, that's the "Churchill" subroutine... ;)

I have a special place for those... I rewrite their requests to subdirectory where all access is bloked except for a custom 403 page. And that custom 403 page is two bytes long... It contains only "no". So, this at least minimizes the bandwidth they waste. Of course, I'd rather block them at the router, but alas, I haven't purchased my own data center yet.

Jim


Thread source:: http://www.webmasterworld.com/search_engine_spiders/3086.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com