Forum Moderators: open
But he does tell malware infested drive-by download sites how to fool it.
[theregister.co.uk...]
...
Love this quote from Roger Thompson, CTO of Exploit Prevention Labs:
"There's so much hacking activity going on the web. The only way to really tell what's there is to go and have a look," he told us. "I don't want to sound flip about this, but if you want to make omelettes, you have to break some eggs."
Which eggs are they trying to break?
IMO, when your anti-hacking tool activity is completely visible to the hackers it's all the AVG customers eggs that are in jeopardy.
However, we've been round and round with this and now it's in the press yet AVG seems not only to not care but is giving flippant quotes.
There are better methods that don't result in any damage being inflicted on bookmarked websites. For instance, one of their competitors has a better solution which involves a transparent proxy and real-time streaming data analysis. It doesn't expose itself to the world, websites don't know it's there, and it still catches the bad sites without spiking traffic needlessly on unsuspecting innocent websites.
[edited by: incrediBILL at 1:24 am (utc) on June 14, 2008]
Webmasters see malicious bots attempting to access their sites on a regular basis. Anything out of the ordinary, or not directly from Google, Yahoo, Live, Ask, is likely to be blocked.
So, where the middle-ground?
AVG are interested in security. They want to scan your site before they rate it as "visitable" for their users
Scan it once a day, 1-Hit from any AVG installation, just once. Store the results on AVG's servers and let the rest of the AVG users get the results from AVG's cache.
What they've done instead is generate a virtual DDoS on some servers that have sites with many bookmarks, 10's of thousands of people bookmark some sites, maybe millions.
So what do you think happens when all those thousands of AVG 8 customers link scan?
That's right, you get a DDoS when all those prior visitors that have your site bookmarked install AVG 8.0!
The only upside is that AVG isn't installed on as many machines as the top AV software or we'd all be in trouble already.
[edited by: incrediBILL at 12:54 am (utc) on June 14, 2008]
AVG are interested in security
Then they should read WebmasterWorld and not The Register.
...
The article says:
1. AVG LinkScanner can be detected by user-agent (details supplied).
2. It can be fooled by feeding it a dummy file (example site given).
3. Further details are available on WebmasterWorld (for those who need them).
I suspect that most malware distributors already know this.
...
The article does not say:
1. Those who rely on AVG LinkScanner for security are being misled.
2. Any malicious site can fool it and deliver them to a drive-by download page.
3. LinkScanner is actually a security risk and makes AVG look incompetent.
I suspect that Grisoft don't understand this yet.
...
[edited by: Samizdata at 12:56 am (utc) on June 14, 2008]
I think that would bring up some privacy concerns. By using log files etc, AVG could, in theory, find out which sites a particular user had visited.
However, this has already created some privacy concerns IMO as many of us now have very large lists of AVG customers by logging all of the bogus user agents. Use your imagination in what could be done if AVG has any gaps in their protection.
If/when AVG switches their software to use the actual user agent, to avoid being detected, we still have that big list of users coming from all those locations so we'll still know where a large majority of AVG users exist on the internet. So simply fixing the user agent won't help anyone who's AV software preference has already been exposed.
At a minimum, I'm sure some other AV software makers or their affiliates would love this targeted list and have probably already started the conversion process.
[edited by: incrediBILL at 1:39 am (utc) on June 14, 2008]
That was going to be my suggestion - a public list of all AVG customer IP's.
a public list of all AVG customer IP's
Google beat you to it Brett - try a search for:
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)
...
Over the next few days, we will be exploring ways in which we can continue to deliver informed protection as unobtrusively as possible without adversely impacting site analytics. Any webmaster reading this post who is interested in working with us constructively to reach this goal is welcome to contact me
My offer of assistance was rebuffed.
...
Roger Thompson earlier this week, he was unaware of these issues. But he defended the role of Linkscanner, which he designed...
... he said. "Given that we've only just been alerted to this situation...
That would be too exensive, you need to maintain a bot, storage, bandwith etc..., right now they are using their client's ressources for this "feature"
If he means what he says he would be posting here
I advised him/her to do so and he/she has certainly read this thread.
...
It's a nice 2-for-1 special, AVG users and webmasters all irritated for one low price.
That would be too exensive, you need to maintain a bot, storage, bandwith etc..., right now they are using their client's ressources for this "feature"
No, a bot can be easily spotted, won't work, trust a bot spotter about this. Using the client machine for a distributed crawl of these sites is much harder to detect and the best way to compile an accurate list of malware sites that isn't easily spoofed.
However, if AVG 8 can actually defend against the malware that the Link Scanner detects then the Link Scanner itself just wasting everyone's time and this feature is just marketing hype with no intrinsic value.
As it is now, a user does a search and gets (typically) ten results. LinkScanner loads those ten results URLs and analyzes them. Now if the user clicks on one of the links in the results, and then later comes back to the same results page, those ten results are scanned again. If yours is the last result on the page and the user goes through and clicks each result in turn, returning to the results page each time, then your page gets loaded and scanned ten times. And this can be even worse if your site has multiple listings in that one results page.
Some have said that they have to do it this way so that the scan results are fresh and up to date. I disagree and would like to point out that the scan is done just after the search results page loads. There is no guarantee that the user will immediately click on any of the link(s) on that results page that he/she is interested in, so some degree of latency is already present in the system.
The main problem they appear to face is the LinkScanner is a "side-car" to the browsing process, rather than a in-line operation. That is, it appears to be part of the Anti-Virus program rather than a browser plug-in. A plug-in approach might make it easier to take advantage of client-side caching, and to use the browser's user-agent instead of the easily-identifiable ";1813" with an IE User-agent.
As stated in one of the other threads here, I'm grateful to AVG for providing a good and free AV solution to those who can't afford to spend $25-$50 per year on AV subscriptions. So let's not beat them up too hard on this until we find out if they will take our interests into consideration now that the problem has been made known. The simple fact is that the additional load on servers, pollution of site statistics, and the security implications of the easily-detected and easily-spoofed user-agent just were not thoroughly considered.
As for LinkScanner building/maintaining a database of bad sites to be kept on a centralized server, that can also be done on a voluntary basis to alleviate privacy concerns -- Let the user opt-in to the reporting and checking feature just like many other such services do...
Jim
It would be very helpful if the client-side LinkScanner simply cached the scanned page contents for any given Search Engine Results Page scan. In this way, our sites are not hit repeatedly each time the user returns to the results page.
That won't solve the sheer volume of hits I'm seeing from thousands of different IPs on a daily basis. It's early in the morning on a slow traffic weekend and AVG's code has already generated over 1K unique IPs hitting the site.
The main problem they appear to face is the LinkScanner is a "side-car" to the browsing process, rather than a in-line operation.
Jim, I respectfully disagree as the main problem is it doesn't need to exist in the first place!
This is an instance where the technology implemented was brute force approach and it's causing major issues for both their customers (see comments on The Register) and webmasters.
My AV program does the same exact page scanning (not link scanning) except it does it in real time using a transparent proxy between my browser and the internet. If something is detected in the data stream it's blocked in real-time, no need to link scan whatsoever, completely transparent.
Why doesn't AVG simply do the same thing and stop annoying everyone?
[edited by: incrediBILL at 3:23 pm (utc) on June 14, 2008]
First, AVG is too dumb to use compression, which means it's taking even more bandwidth.
Secondly, AVG doesn't only precharge the page that Google proposes (example.com/interesting.html), but ALSO the site index page (example.com/) - even though Google does not propose / in the results in question.
Thirdly, AVG often (but not always) doesn't understand example.com/interesting/ and leaves off the final /, and so gets a 301 that it follows.
These log entries show the first two points, AVG downloads the page that Google proposes and the site index page without compression, then the user dowloads the page that Google proposed with compression:
90.16.11c.dd - - [14/Jun/2008:08:22:29 +0200] "GET /aaa/bbb/interesting.html HTTP/1.1" 200 46506 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)"
90.16.11c.dd - - [14/Jun/2008:08:22:29 +0200] "GET / HTTP/1.1" 200 8365 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)"
90.16.11c.dd - - [14/Jun/2008:08:22:31 +0200] "GET /aaa/bbb/interesting.html HTTP/1.1" 200 15655 "http://www.google.fr/search?hl=fr&q=%22blabla%22&btnG=Recherche+Google&meta=" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.04506)"
Here are the logs for the third point (the 301), which may only happen with Yahoo:
79.88.25c.dd - - [13/Jun/2008:10:36:08 +0200] "GET /aaa/interesting HTTP/1.1" 301 259 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)"
79.88.25c.dd - - [13/Jun/2008:10:36:08 +0200] "GET /aaa/bbb/other.html HTTP/1.1" 200 84606 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)"
79.88.25c.dd - - [13/Jun/2008:10:36:08 +0200] "GET /aaa/interesting/ HTTP/1.1" 200 25936 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)"
Peter
PS. [Edit] This suggests that a 301 to AVG's own site would solve the ;1813 problem nicely, but I haven't tried it!
[edited by: Peter at 3:41 pm (utc) on June 14, 2008]
This suggests that a 301 to AVG's own site would solve the ;1813 problem nicely, but I haven't tried it!
I've had one in place since this tongue-in-cheek mention:
[webmasterworld.com...]
It hasn't slowed the requests down a bit, nor has the 301 eliminated successive requests for the same page.
let's not beat them up too hard on this until we find out if they will take our interests into consideration
I have no interest in beating up AVG - but I can't help laughing at them.
WebmasterWorld members have been highlighting the LinkScanner problems for more than a month, and some have tried to help the company by contacting them directly - only to be met with complacency (at best) or something very close to offensiveness.
I sincerely hope that others who respond to Pat Bitton's cry for help can get through to them.
Even if the company ignores our interests, they surely need to consider their own.
...
As stated in one of the other threads here, I'm grateful to AVG for providing a good and free AV solution to those who can't afford to spend $25-$50 per year on AV subscriptions - jdMorgan
Jim
My own free copy (single machine, non-commercial) of AVG would no longer download virus defs, announcing that the free version had been discontinued and that to continue to protect my blah blah I would need to upgrade.
I wasn't aware I could have downloaded/installed another copy. Betcha it turns into a bait'n switch.
Betcha it turns into a bait'n switch
What is has turned into is a farce.
Like many other packages AVG offers a free version and uses it to encourage people to buy the paid version - nothing wrong with that, and AVG anti-virus itself works very well and has deservedly become very popular.
Whether the popularity will survive the LinkScanner fiasco remains to be seen.
If AVG's "head of egg-breaking" keeps his job it won't deserve to.
Our primary responsibility is to provide the best possible protection for our users
Mr Thompson, identifying the IP addresses of AVG users in every site log on the web - some of which are publicly available - is a gross dereliction of your primary responsibility and you should publicly apologise for it.
In addition, LinkScanner is so easy to fool that it is the very opposite of a security tool, and in promoting it as "protection for our users" your company is not only selling snake-oil but is also putting its customers at risk.
I don't know what Grisoft paid for LinkScanner or what they are paying you, but they would have been much better off putting all the cash into a big pile and setting fire to it.
...
According to Thompson, nearly all web exploit toolkits track IP addresses
According to Samizdata, scammers will now have the IP addresses of many AVG users.
Perhaps someone who still has a job at AVG can confirm this?
Even a "no comment" would be civil.
...
[edited by: Samizdata at 4:11 am (utc) on June 16, 2008]
But 48 hours have passed since AVG told me there was nothing in this thread that interested them and I was wondering whether they might have changed their opinion at all in that time.
I will, however, take your advice and wait for the official statement.
...
Maybe they'd be interested to know I'm getting ready to drop about two dozen paid subscriptions over this crap.
Yes, you can turn it off, but then you get to look at an ugly logo in your try because you turned this off and aren't "fully" protected.
