Welcome to WebmasterWorld Guest from 107.20.59.213

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

AVG Toolbar Glitch May Be Causing Visitor Loss

User Agent Flaw Suspected

     

Umbra

2:36 pm on Mar 31, 2008 (gmt 0)

10+ Year Member



Seeing a rash of hits with an oddly formed user agent:

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)
No referer

mod_security always throws an error for this one. Hits come from various IPs with no consistent pattern, seem to be residential IPs. Any idea what it is?

jdMorgan

11:37 pm on May 11, 2008 (gmt 0)

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member



As suggested in several posts earlier in this thread, returning a very small valid html page to the AVG Linkscanner client is a much safer way to conserve bandwidth without risking traffic or revenue loss. In .htaccess:

RewriteCond %{HTTP_USER_AGENT} ;1813\)$
RewriteRule !^a-very-small-page\.html$ /a-very-small-page.html [L]

Jim

blend27

12:09 am on May 12, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



and as far as AdWords goes on the same page, 8 out of 10 Advertisers had a big grey question mark next to the Ad, including EBAY, HSN, JTV and other big PLAYAS in a given niche, so there goes CTR...

smallcompany

12:47 am on May 12, 2008 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



Funny, I haven't seen anything except green checkmarks. I just queried ebay and hsn and got all greens. Specific to a landing page, right?

incrediBILL

1:02 am on May 12, 2008 (gmt 0)

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Funny, I haven't seen anything except green checkmarks.

The pages would most likely have to contain a virus injector or phishing code to cause the toolbar to signal an alert.

I have a bunch of URLs that *should* set it off but I can't post them here, but I'll post the results when I get around to testing them.

Samizdata

1:16 am on May 12, 2008 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



403 puts a BIG green check mark to the right of the listing in SERP

My tests consistently say 403 = grey question mark (and no approval from AVG).

My 403 is triggered by an NT version-checking routine that expects a space after the semi-colon.

Apache responded this way to 29 identical requests in 9 seconds.

Meanwhile I confirm that I was not pressured into allowing access to this user-agent.

Grisoft simply made me an offer I couldn't refuse...

incrediBILL

5:44 pm on May 13, 2008 (gmt 0)

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Been watching this toolbar in my log files and it's added 7,053 to my page views so far this month and it's escalating daily.

Here's the last few days of AVG toolbar traffic in order: 592, 662, 720, 905, anyone else seeing this growing trend?

wilderness

6:20 pm on May 13, 2008 (gmt 0)

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Wonder if it's possible this increase has anything to do with the recent release of SP3 update for XP?

smallcompany

7:45 pm on May 13, 2008 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



Isn't increase related directly to the fact that people are upgrading/installing new AVG 8.0?
Previous AVG was not doing this link check.

All this with the assumption that it is AVG, both 1813 and SV1 UAs.

Samizdata

8:22 pm on May 13, 2008 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



anyone else seeing this growing trend?

I am - though no AVG installations that I upgrade myself are getting the toolbar, obviously.

possible this increase has anything to do with the recent release of SP3 update for XP?

In my (albeit uneducated) opinion it is entirely unrelated to SP3.

All this with the assumption that it is AVG, both 1813 and SV1 UAs

I do not relate the term SV1 to the AVG toolbar user-agent at all.

Can we expect similar from Symantec, MacAfee et al, or do we already have them?

I asked this question earlier - can anyone answer it?

--

I also notice from this thread [webmasterworld.com] that AVG is having fun banning innocent sites on shared IPs.

[edited by: encyclo at 10:42 pm (utc) on May 13, 2008]
[edit reason] fixed link [/edit]

dstiles

2:24 am on May 14, 2008 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



Further to an early mention of the UA...

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

The pattern I'm seeing for this is extremely similar to the 1813 case - single IPs, missing ACCEPT and no referer. It also almost invariably has some a querystring tracer missing, which in this case is indicative of it coming from a search engine and most certainly not from elsewhere within the site.

It seems possible to me it could either be an earlier version of an AVG signature, possibly from another OS, or perhaps from another AV/Firewall company.

smallcompany

3:35 am on May 14, 2008 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



On my sites, Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) behaves absolutely the same as 1813.

I actually asked about this one back in March. That is when I first noticed its strange behavior, but wasn’t aware of its background. That is why I created a post in Browsers section:

[webmasterworld.com...]

Vamm

8:09 am on May 14, 2008 (gmt 0)

5+ Year Member



The increase is not related to SP3.

The reason is
AVG 7.5 (with NO toolbar) is currently displays banners stating version 7.5 is end-of-life. Although deadline is declared 31/12/2008, people already starting to upgrade (to AVG 8.0 with toolbar), because of the banner.

Samizdata

1:29 pm on May 14, 2008 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



Jim was right (as usual).

This user-agent does not come from the "Security Toolbar" (which is an optional, if pre-checked, install) but from the AVG LinkScanner component (which is installed by default).

I upgraded another three AVG installations today, each without the toolbar, and all had their search results interfered with by the AVG internet police, wasting bandwidth and leaving the tell-tale footprint Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813) (which as we have seen is easily fooled, making it useless or worse).

Once again, sites that gave it a 403 were "greylisted" and given an effective thumbs-down.

Grisoft, you just jumped the shark.

mindaugas13

8:57 pm on May 14, 2008 (gmt 0)

10+ Year Member



I too installed AVG Free 8.0 and verified that it visits sites with this user agent:

mozilla/4.0 (compatible; msie 6.0; windows nt 5.1;1813)

This is actually done by the LinkScanner feature of AVG 8.0. When I installed AVG, I specifically did not install the Security Toolbar (option during custom installation). However the Link Scanner is part of the main AVG program. It can be disabled, but then the AVG icon appears as a red exclamation mark.

johnnie

12:36 am on May 15, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



By the way, in IE the search result-scanner can be easily disabled through internet options - > programs -> add-ons. Just disable the safe search. Seriously, who makes up this crap...

smallcompany

2:19 am on May 15, 2008 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



Since AVG is on the famous highway towards becoming bloatware, we can only hope for a massive shift towards avira.

AVG did enough to promote itself. I did grab Avira recently as a part of my AV testing, but how many of ordinary people care abut this? None.

By the way, in IE the search result-scanner can be easily disabled through internet options - > programs -> add-ons. Just disable the safe search. Seriously, who makes up this crap...

Continuing from the above, an average user never goes to any of the options under menu.

It is just that all of these companies are trying to stay in the game by inventing something “new”, something that (like) adds an extra protection layer.
Most of that turns to be a marketing move, and nothing else.

That is why I took long time good AV off my machine and had it (my PC) run like never before.

Back to the topic… It is about all these companies, including Grisoft, to ensure they don’t interfere with other people’s business, in any meaning. If they want to do something like this, they better do it transparently.

spotter

5:43 pm on May 15, 2008 (gmt 0)

5+ Year Member



I’ve been watching this thread with interest and finally had to chip in. Check out FAQ 1338 on the Grisoft site and it gives you a command line call to install AVG8 without the Linkscanner. You can indicate that the FAQ was helpful, so if enough of us do it they may get the message, even if you have no intention of using AVG.

Apparently Grisoft purchased Exploit Prevention Labs to acquire this useless piece of software. Guess it was cheap.

I’d been noticing hits from the product in my logs for some months but could not figure what they were until it was incorporated into AVG. I have an external js function with a variable passed in as a parameter and called with +<varname>+. This trips it up and it returns a 404 with the code snippet rather than the filename. The site still shows as approved in the SERP.

System

3:21 am on May 16, 2008 (gmt 0)

redhat



several messages were cut out to new thread by incredibill. New thread at: search_engine_spiders/3651560.htm [webmasterworld.com]
9:45 pm on May 15, 2008 <small>(PST -8)</small>

[edited by: jatar_k at 12:12 pm (utc) on May 16, 2008]

smallcompany

7:27 pm on May 23, 2008 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



...but this thread is still about 1813, right?

incrediBILL

7:40 pm on May 23, 2008 (gmt 0)

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Yes, all about ";1813" ;)

blend27

3:15 pm on May 24, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I wonder if the number has anything to do with "sic itur ad astra"

Umbra

6:22 pm on May 24, 2008 (gmt 0)

10+ Year Member



Once again, sites that gave it a 403 were "greylisted" and given an effective thumbs-down.

How does AVG respond to an error 500 response?

RSweetnam

4:45 pm on May 29, 2008 (gmt 0)

5+ Year Member



I've been seeing similar behaviour on my site although the really strange this is that all the IP address where I see the useragent from are all located in my home country and from different ISP's.

[edited by: incrediBILL at 5:23 pm (utc) on May 29, 2008]
[edit reason] URL removed, see TOS #13 & #25 [/edit]

jdMorgan

5:31 pm on May 29, 2008 (gmt 0)

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member



I've been using the "deliver a small, simple page with one link" method since the day we figured out this was AVG LinkScanner. I've had no problems, but wasted bandwidth is way down. :)

The link on my short/sweet page just links to my home page -- It doesn't seem to need to be any specific link.

Jim

superclown2

11:38 am on Jun 1, 2008 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member Top Contributors Of The Month



Jim, please excuse what may sound like a very uneducated query but I confess that mod-rewrite scares the h*ll out of me, so when you write:

RewriteCond %{HTTP_USER_AGENT} ;1813\)$
RewriteRule !^a-very-small-page\.html$ /a-very-small-page.html [L]

I appreciate that the second 'a-very-small-page.html' refers to the URL of a, well, very small page, but what do I change 'a-very-small-page\.html$' to, please?

Samizdata

2:39 pm on Jun 1, 2008 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



It should be the same filename, as this prevents creating an infinite loop.

The rule says "if the request is NOT for the substitute file, serve the substitute file".

The dot before the extension needs to be escaped in the first part of the rule.

superclown2

8:46 pm on Jun 1, 2008 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member Top Contributors Of The Month



Thanks, I'm very grateful but I can't get it to work. I created a file called block.html and this is my .htaccess file:
RemoveHandler .html .htm
AddType application/x-httpd-php .php .htm .html
RewriteEngine on
RewriteBase /
RewriteCond %{HTTP_USER_AGENT} ;1813\)$
RewriteRule !^block\.html$ /block.html [L]

Please, any suggestions about what I'm doing wrong?

Samizdata

9:56 pm on Jun 1, 2008 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



Much depends on how your server is set up and what your other directives are supposed to do, but I would say the likely cause of your problem is the RemoveHandler directive.

Why not try naming your substitute file block.php instead?

# Set options (may be required)
Options +FollowSymlinks

# Turn on mod_rewrite
RewriteEngine On

# Deal with idiotic prefetch
RewriteCond %{HTTP_USER_AGENT} ;1813\)$
RewriteRule !^block\.php$ /block.php [L]

superclown2

7:45 am on Jun 2, 2008 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member Top Contributors Of The Month



Ah! This is the reason, the user_agent is different:
"GET /my-web-page.html HTTP/1.1" 200 32961 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"

What is user agent SV1? Should I be rewriting that instead or will that kill legitimate traffic?

I'm amazed that this company seems perfectly happy to screw up the whole world's web stats, this could develop into something very interesting.

superclown2

11:38 am on Jun 2, 2008 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member Top Contributors Of The Month



"What is user agent SV1? Should I be rewriting that instead or will that kill legitimate traffic?"

Sorry I really need to learn to read a thread properly without asking questions that have already been answered.

Leaving aside the mess in all our stats and the inevitable trouble that this will cause Grisoft when all those angry webmasters realise who is responsible for it I really can't see the need for this pre-loading when Google already label dangerous sites, and since I installed the toolbar on one of my high-spec computers with 16 meg broadband the load time for Internet Explorer has increased to about 15-20 seconds so I for one have removed it and put Norton back on. The sooner they drop this pile of xyz the better for their business as well as ours.

This 173 message thread spans 6 pages: 173
 

Featured Threads

Hot Threads This Week

Hot Threads This Month