Welcome to WebmasterWorld Guest from 188.8.131.52
Forum Moderators: phranque
I found out something new about AVG today. There is an option that is reportedly checked by default on the new 138 version. You can see it under "Advanced Options." It's called Navigation Error Redirection: "If you happen to navigate to a non-existent webpage, AVG can provide you with suggestions to assist you in finding the page you are looking for."
AVG then intercepts 404 results and substitutes their own page. This is not a helpful page, because while it's vaguely relevant to the keywords in the URL, every single one of the links on AVG's "helpful redirect page" is a pay-per-click ad link, according to screen shots that someone sent me.
AVG explains this page here:
[edited by: incrediBILL at 2:05 am (utc) on July 16, 2008]
[edit reason] fixed formatting and link [/edit]
Anti-Spyware: protection against spyware, adware and identity-theft
Assuming they are inserting pages of ads on 404 errors, then the protection against adware has itself become adware!
I'm sure the irony isn't being lost on anyone here...
So now it seems that the AVG "DNS/404 Redirect" [grisoft.com] feature, which is enabled by default, is potentially exposing your search habits to a third party by default without your express permission.
Does anyone see any privacy issue here?
Sounds like a little more research needs to be done to fully investigate this issue but this feature should definitely be an opt-in feature, not opt-out, by default.
[edited by: incrediBILL at 2:41 am (utc) on July 16, 2008]
this might make a webmaster think about returning anything except a 404 if the user agent has the SV1.
I think you miss the point, the SV1/1813 days are over, LinkScanner no longer does that.
This appears to be in AVG itself, their Surf Shield, if you hit a 404 it does a redirect according to their docs.
Search Shield is in both the paid and free versions. That's the one that operates on links found on Yahoo, Google, and Live search-results pages.
The LinkScanner component in AVG, in both paid and free versions, is disabled for website prefetching, but it still does a local DNS lookup and pretends that this is just like the old component. However, AVG owns linkscanner.com, and the Lite version of the stand-alone LinkScanner, still available for free on that site, is still doing the website prefetching. Their user base is small, fortunately. The .htaccess files recommended widely to catch the old AVG will also catch the LinkScanner Lite prefetches.
I believe that the 404 hijacking occurs on Search Shield as well as Surf Shield, according to information I received. I'm going to do some testing today on the free 138 version of AVG and see if I can get the 404 hijack to work. If I can, I'll throw Wireshark on it and see what it sends to AVG. I'm curious about whether the registration info is sent to AVG, and I'm curious about whether any cookies are left by AVG, and I'll also try it on a custom 404 page to see if it replaces that, as well as replacing a standard browser 404 page.
This "feature" is related to the toolbar. If you see the AVG toolbar on your browser, the 404 hijack is enabled unless you specifically disabled it.
At the bottom of the new 404 page, you can click to disable this feature. Otherwise, it is hard to find. On the left side of the AVG toolbar, there is the AVG logo, the big letters AVG, and a tiny down arrow next to that. Click on the down arrow, click on Options, click on the "Advanced" tag, and then unclick on the 404 redirect option.
It's not the sort of redirect we do with htaccess, but rather an internal interception of the the browser data stream that forces a new GET. It happens on all 404 pages, whether or not you found the link on a search results page, or keyed the link directly into your address bar. The interception happened fast on my Firefox, but on my Explorer is was slow enough so that I could see my custom 404 page for a second before it kicked over to the intercept page.
The whole thing is handled by vmn.net. The address of the new 404 page as well as most of the PPC links on that page, starts out as avg.urlseek.vmn.net. There is a search string after that. It happens on browser 404 pages as well as your custom 404 pages.
The search at vmn.net is keyed on the URL alone, which means the matching is rather poor. You can see what the replacement page will look like by doing this:
Substitue the query string with your own domain and a page. This will show you how the AVG replacement page will look, except it will say "404 Error" in the upper right corner instead of "DNS Error."
The "Powered by Yahoo" logo next to the new search box on that page merely searches vmn.net again. I feel that this logo is misleading, because a vmn.net search is merely a search of their ad inventory, not the web. (The AVG page says "Search" instead of "Web Search" and the form is at the top of the page.) In addition to all the extra urlseek.vmn.net links to PPC ads, there may be some "sponsored links" that go to overture.com.
The option to turn off the "redirection" at the bottom of the AVG page goes right to the "Advanced" option box for the AVG toolbar.
If you search for "urlseek.vmn.net" on Google, you will find links from people who get stuck on this page and don't want to be there. Apparently urlseek.vmn.net is used on a variety of toolbars, which probably intercept various status codes in addition to a 404. Most people consider it malware.
You get all the cookies you expect from urlseek.vmn.net. Also, if you don't have the unique ID cookie from www.avg.com already, you get it as soon as you load your browser with the AVG toolbar. It is forced into the browser cookie file directly. I don't think it is being used right now, because www.avg.com is redirected by AVG to www.grisoft.com. It expires in one year.
Of the several vmn.net cookies, the longest one expires in two years. It looks like they have unique IDs in them. Note that the fully-qualified domain name starts out with "avg.urlseek.vmn.net." Google Analytics is also active at avg.urlseek.vmn.net, according to the headers I captured. The reports of AVG activity are probably handed over to AVG. I don't know if AVG can trace these to a specific toolbar installation, other than through your IP address.
So does the AVG interception mean that my redirected page is dumped in favour of AVG adverts?
Sounds like a variation on PHORM to me!
If a 404 status is seen by your browser at any point during this fetch, then I suspect the answer is that the AVG user will be sent to the adware page within a second or two.
AVG Redirect turned on:
If the original web site has a 404 override (eg sends it back to the home page) the site's redirect is honoured (at least in the test we made).
If there is no redirect AVG goes to VMN and provides sponsored adverts (in this case only one, probably because that's all there was that was remotely connected to the topic) plus a lot of links heisted from somewhere else (MSN? That's a guess if they are in business with MS).
AVG Redirect off:
If the site normally returns an alternative page then that is still returned, otherwise a more traditional 404 message; ie it works as per normal expectation.
So, basically acceptable as far as webmasters are concerned but, as the (internet-savvy) tester mentioned to me, a bit dodgy on the unsuspecting novice who thinks the links are all that's available on the subject.
basically acceptable as far as webmasters are concerned but
Not really because now I have to be concerned that any hard 404's I leave lying around without redirects potentially expose my visitors surfing habits to a 3rd party which has this feature enabled by default.
I'm not sure but it sounds like some privacy issues and who takes the fall for that liability, AVG or the webmaster that didn't know his hard 404 errors were exposing his customers to AVG's tool?
I suspect by far the majority of sites do not include managed 404's and most cannot even modify their server's error set in any case. I agree AVG should not try to hijack in this fashion.
Although don't some browsers auto-redirect to google/msn (toolbars?) anyway on a 404? Not so commercially sensitive and more obvious but a similar situation.