homepage Welcome to WebmasterWorld Guest from 54.161.175.231
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

This 39 message thread spans 2 pages: < < 39 ( 1 [2]     
AVG Stops Real-Time Scanning
Webmasters Complain AVG Debilitating Traffic Analytics
Cyclob




msg:3691943
 5:44 am on Jul 7, 2008 (gmt 0)


System: The following message was cut out of thread at: http://www.webmasterworld.com/search_engine_spiders/3674410.htm [webmasterworld.com] by incredibill - 11:46 pm on July 6, 2008 (PST -8)


Good News folks! Seems like AVG stopped this real time scanning since they announced on July 5, 2008 in wikipedia... as follows...

[en.wikipedia.org...]

LinkScanner concerns:

The LinkScanner feature has been shown to cause an up to tenfold increase in traffic on web sites that appear in a search engine's search results. Since LinkScanner disguises itself as a click by a human being when it prescans each site listed in the results, web site usage logs will show incorrect and overinflated site visitor statistics. The prescanning of links also causes web sites to transfer more data than usual, resulting in higher bandwidth usage for users and web site operators.[8] AVG says site administrators will be able to filter the LinkScanner traffic out of their site statistics, leaving the problem of excess bandwidth usage still to be solved.[9] Pay-per-click advertising is not affected by the increase in traffic.[10]

On 2008-07-05, AVG announced that as of 2008-07-09 "Search-Shield will no longer scan each search result online for new exploits, which was causing the spikes that webmasters addressed with us."[11]

[edited by: incrediBILL at 7:44 am (utc) on July 7, 2008]
[edit reason] formatting cleanup [/edit]

 

Scarecrow




msg:3699530
 5:21 am on Jul 16, 2008 (gmt 0)

It's nowhere near over as this LinkScanner is still being sold online as a stand alone product and there's no mention whether or not it no longer hits every site.

I downloaded the free LinkScanner Lite today from the linkscanner.com site. It is version 2.7.0. Yes, it's still fetching all the links shown in Google, Yahoo, and Live.com, and it doesn't do any caching. It uses the SV1 user-agent. The only saving grace is that LinkScanner Lite has a tiny user base compared to AVG.

g1smd




msg:3700228
 9:07 pm on Jul 16, 2008 (gmt 0)

I installed AVG the other day, the installer filename ended in 138__1332 or somesuch.

As it was being installed, AVG still offered Linkscanner as an available option.

We deselected it, of course.

tangor




msg:3700537
 5:51 am on Jul 17, 2008 (gmt 0)

Don't use javascript, don't require it for my site, and don't use Google Analytics. Too many disable javascript and I'd rather not lose those visitors. I do agree that raw logs only is not the best of worlds, but it is the world I have and am willing to support.

incrediBILL




msg:3719989
 11:10 pm on Aug 10, 2008 (gmt 0)

I'm still getting a bunch of hits from ";1813" so it appears the update wasn't automatic or this still wouldn't be happening.

Samizdata




msg:3720002
 12:04 am on Aug 11, 2008 (gmt 0)

I still see a handful each week, but nothing like the original onslaught.

Some will be installs from archived copies, others might be due to System Restore.

I believe the old software required at least two updates to get the new castrated version.

And we all know how to deal with it, of course.

Your post prompted me to check [blogs.avg.com...] again, which I found amusing.

...

dstiles




msg:3720025
 12:56 am on Aug 11, 2008 (gmt 0)

Still getting about 80 a day, usually between one and 3 per IP, on a single-site server with a fair degree of traffic and about 100 a day on a low-traffic multiple-site server.

Some people do not permit automated updates, sometimes for good reasons, such as running from dialup, not wanting an automatic reboot in the middle of doing something vital, whatever.

To begin with, AVG trumpeted about updating to V8. I doubt they have trumpeted about updating to the latest version - or at least, not why and some people want a good reason. So people who control their software updates wouldn't know to upgrade. As long as the signatures keep coming in...

incrediBILL




msg:3720034
 1:23 am on Aug 11, 2008 (gmt 0)

What blog? It's empty! LOL!

keyplyr




msg:3720122
 8:49 am on Aug 11, 2008 (gmt 0)

What blog? It's empty! LOL!

Probably a virus detected.

Scarecrow




msg:3725102
 9:52 pm on Aug 17, 2008 (gmt 0)

I've been tracking AVG LinkScanner hits since AVG disabled LinkScanner in their July 9 update. I've discovered that my LinkScanner hits are tremendously dependent on my ranking for what I call a "newbie search."

A newbie search is done by someone with close to zero familiarity with PCs, the web, and search engines. They use Google by entering a single keyword, and rarely go past page one of the results (with the default setting of 10 results per page).

I have two sites that are particularly susceptible to LinkScanner. Since late June, one has ranked between 15 and 35 in Google for the single keyword "gmail." The other site has a home page that has ranked nowhere in Google for the single keyword "wikipedia" -- until a few days ago, when it popped up to rank 9 in Google (page one of the results) for that single term, and after a couple of days settled into rank 11 (page two of the results), where it remains today and will probably disappear by tomorrow. (It took almost three years for that anti-Wikipedia site's home page to get some honest recognition from Google, despite the fact that Yahoo has reported 3,400 external backlinks for that home page for over a year, and it has shown a PageRank of 5 in Google for 14 months now.)

Anyway, the point is that with both the gmail and the wikipedia keyword, the respective site will essentially double in LinkScanner hits as it crosses the border between a page three on Google results to page two, and more than double again if it passes from page two to page one of Google results. The dumber, more generic, and more "newbie" the single-word search term that's used, the more this effect is evident.

Anti-virus software packages are attractive to people who don't know much about the web, and believe that these packages will keep them safe. A lot of them, I suspect, are installed for grandpa or grandma by their grandson, and they don't get updated like they should because the grandson doesn't follow up.

The life-cycle of anti-virus packages would make an interesting study. They thrive on hype, and they constantly need more gadgets in their packages to make them look like they're better than the competition. In terms of performance, about all they do is slow down the computer and occasionally catch something that is probably unimportant, just so that they can look like they're hard at work.

It's been amazing to me that AVG is able to fake it with LinkScanner and save all the hype they've invested in their LinkScanner module, just by continuing to do the DNS lookup without the site scan. The DNS lookup alone tells you nothing about how safe a site is, due to all the sites that share a single IP via name-based hosting. But it still looks like you're scanning all the links out there in real time, which is impressive for anyone who doesn't know anything about DNS, and it's quite fast.

And AVG is getting away with this. All you have to do is fool enough newbies, and that's good enough for anti-virus packages.

This 39 message thread spans 2 pages: < < 39 ( 1 [2]
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved