Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- WebSense Hiding Behind Rotating User Agents

wilderness - 9:45 pm on Jul 10, 2008 (gmt 0)


System: The following message was cut out of thread at: http://www.webmasterworld.com/search_engine_spiders/3695122.htm [webmasterworld.com] by incredibill - 2:16 pm on July 10, 2008 <small>(PST -8)</small>

Yankee Doodle ;)

208.80.193.* - - [10/Jul/2008:22:06:01 +0100] "GET / HTTP/1.0" 301 391 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; {C13455BA-974A-C95C-848E-98CF3CA8035D}; .NET CLR 1.1.4322; .NET CLR 1.0.3705)"
208.80.193.* - - [10/Jul/2008:22:06:01 +0100] "GET / HTTP/1.0" 200 8040 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Wanadoo042NLPP01HCFFFFFFE8; SV1; SIMBAR={2B30982A-505F-4a6c-8941-2F678BA39B6B})"

I've pretty much left websense alone. They generally visit multiple times each day and only grab the sites main pages.

This the last draw.

deny from 208.80.192.0/21

[edited by: incrediBILL at 10:18 pm (utc) on July 10, 2008]
[edit reason] Obscured IPs [/edit]

Thread source:: http://www.webmasterworld.com/search_engine_spiders/3692641.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com