Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- a legitimate blank?


wilderness - 8:48 am on Jun 9, 2011 (gmt 0)


keyplr,
FWIW, the first two RewriteCond's g1smd provided must exist together (double conditions of both blank refer and blank UA), were they separate, the first line would end with an [OR]

I don't block blank refers on their own, however have some lines where I utilize that with multiple conditions.
EX:
Blank refer from a specific IP
Blank refer from a specific UA
or any combination.
One could even include header checks.

Any general visitor that goes browsing with both blank refers and blank UA's certainly deserves denying, despite their lack of awareness ;)


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4303131.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com