Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Hits from unidentified blog trackers


GaryK - 4:04 pm on Sep 27, 2006 (gmt 0)


BOING! BOING! BOING!
That's the name of the new BoingBoing podcast. :)

Your rants are always the best Bill!

On a serious note, I don't even bother with the slash like you did. I suppose if a crawler came around like, for example, Conjavabot, which I just made up, they'd be banned under my rules. So perhaps for those webmasters who prefer a more conservative approach your pattern is a better example than mine.

Then again mine will catch these nasty and persistent little user agents so maybe it's not so bad after all:

Java(TM) 2 Runtime Environment
Java1.1.7
JPluck/2.0.9 (Java 1.4.2_03; Windows XP)


Thread source:: http://www.webmasterworld.com/search_engine_spiders/3097217.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com