Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- WebSense Hiding Behind Rotating User Agents


Samizdata - 12:31 am on Jul 11, 2008 (gmt 0)


Websense ThreatSeeker Network scans over 595 million websites per week, searching for threats.

They only try to take the home page (no scripts or other files) and use many user-agents.

They attempt to access my sites (and yours) by deception in order to make their profits.

Websense Web Security Suite is a leading web security solution that protects organizations from known and new web-based threats.

Their methods are both dishonest and (in my case) hopelessly ineffective. But...

"No one is this world, so far as I know - and I have researched the records for years, and employed agents to help me - has ever lost money by underestimating the intelligence of the great masses of the plain people." (H L Mencken)

...


Thread source:: http://www.webmasterworld.com/search_engine_spiders/3692641.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com