-- Search Engine Spider and User Agent Identification
---- AVG Stops Real-Time Scanning
Cyclob - 5:44 am on Jul 7, 2008 (gmt 0)
System: The following message was cut out of thread at: http://www.webmasterworld.com/search_engine_spiders/3674410.htm [webmasterworld.com] by incredibill - 11:46 pm on July 6, 2008 (PST -8) Good News folks! Seems like AVG stopped this real time scanning since they announced on July 5, 2008 in wikipedia... as follows...
The LinkScanner feature has been shown to cause an up to tenfold increase in traffic on web sites that appear in a search engine's search results. Since LinkScanner disguises itself as a click by a human being when it prescans each site listed in the results, web site usage logs will show incorrect and overinflated site visitor statistics. The prescanning of links also causes web sites to transfer more data than usual, resulting in higher bandwidth usage for users and web site operators. AVG says site administrators will be able to filter the LinkScanner traffic out of their site statistics, leaving the problem of excess bandwidth usage still to be solved. Pay-per-click advertising is not affected by the increase in traffic.
On 2008-07-05, AVG announced that as of 2008-07-09 "Search-Shield will no longer scan each search result online for new exploits, which was causing the spikes that webmasters addressed with us."
[edited by: incrediBILL at 7:44 am (utc) on July 7, 2008] [edit reason] formatting cleanup [/edit]