Welcome to WebmasterWorld Guest from 18.104.22.168
But he does tell malware infested drive-by download sites how to fool it.
My server is being overloaded by this stupid new "feature" for AVG. I think people should continue to notify AVG about this problem, so they can fix it.
My DB is causing a huge increase of server load and I got 2 Ddos already
. Can someone please send me a .htaccess code to redirect user agent
"User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
to a static html page?
And thanks also for the entertainment value.
its illegal what they are doing
IANAL but I don't know of any law against being ridiculous or stupid.
I propose that their next user-agent should be:
LinkScanner (Incompetent; Dishonest; Wasteful; Easily Fooled; AVG 2008)
Funny thing is, that would not trip any current filters on my sites.
I feel another RewriteCond coming on...
There should be a popup when you mouse over the link.
It should say "For additional information click here".
I advise you to click there and start taking screenshots.
Then check your site very carefully for any code or files you did not put there.
If you are sure your site is clean contact AVG.
While you are waiting for a reply read this thread.
Is there any chance that by blocking the fakes that actual customers are being blocked?
I am confused as to whether it should or should not be blocked...is there a consensus on what should be done?
I would love to stop the bombardment but by the same token I do not want to take the chance of losing any customers either because of the block or the chance that the block may cause me not to get the green check mark...
This is so confusing.....
I am still getting bombarded with requests for files
You cannot stop the requests - they are made automatically every time your site appears in the search results of a LinkScanner user, so some sites can expect thousands a day.
Example: if a LinkScanner user searches for "widget" and the word appears anywhere on your site there is a good chance you will see a hit, even if the kind of widget your site mentions is completely unrelated the the kind of widget that was being searched for.
Therefore almost any well-ranked site will be plagued by LinkScanner.
Is there any chance that by blocking the fakes that actual customers are being blocked?
Firstly, blocking with a straight 403 is not a good idea - LinkScanner responds with repeated requests (in one case I had 120 in 12 seconds), and will mark your site as "questionable" (which will naturally discourage people from visiting it).
What many here have been doing for the past six weeks is either serving a very small file to LinkScanner or redirecting it to AVG's site. In both cases their site is marked as clean. If done correctly I would say there is no chance that actual customers are being blocked.
Various examples of how to do it have been posted already, but as AVG recently introduced some new (and very obvious) dishonest user-agents the examples may have to be amended slightly.
This is extremely easy for those who understand .htaccess (or the Windows equivalent) but those who are new to it really need to do some reading and learning as simply copy/pasting .htaccess that you don't understand is a recipe for disaster.
This is so confusing
A reasonable comment, but if you read WebmasterWorld you should achieve enlightenment.
And you will be a lot less confused than AVG, who are apparently clueless.
Once again, I strongly encourage any webmaster who is experiencing problems due to LinkScanner to contact AVG with specific details and examples. Be polite, be patient, and do not rant.
I'm finding this agent is crawling through some of the pages without any cookie control, and not able to read URLs correctly either. In my case it was ignoring the terminating quote in a URL.
Where its become a problem on some of my websites, I've outputted just this to the user agent:
Anyone think this was a bad idea?
Any suggestion on how AVG non-subscribers go about contacting AVG about these issues?
You only need to type three letters into your favourite search engine to find a selection of postal addresses, phone numbers and fax numbers in a variety of countries (no email addresses though).
If you click the link in the first post in this thread and look at the comments on the article in The Register you will see the email address of AVG's head of communications under a specific request for webmasters to get in touch.
Once again, please be polite, I suspect they may be feeling rather sensitive.
...which will allow us to continue to provide the best possible protection for our customers, without imposing too much extra bandwidth on websites
I'm sure the world wide bandwidth meter has made a jump. ISPs everywhere are probably wondering whats going on.
Already had 10K hits by noon just by AVG and it's a slow Saturday.
Not a happy camper.
For Good Guys:
Dishonestly poses as a human visitor
Does not adhere to established robot protocols
Marks unexamined sites as "questionable" and discourages visitors
Wrongly blacklists some sites with shared IPs as malware sites
Does not inform sites they have been blacklisted
Cannot handle a simple server response e.g. 403
Makes numerous repeated requests unnecessarily
Makes incorrect and unnecessary HEAD requests
Fails to use caching
Wastes significant bandwidth
Renders some analytics techniques unusable
Possibly disables "first click free" techniques
Tries to deceive webmasters
For Bad Guys:
Easily fooled by malware sites
Uses no bandwidth if redirected to AVG by malware sites
Identifies user IP to malware sites
Vulnerable to placement of identifying cookies by malware sites
Vulnerable to delivery of drive-by downloads by malware sites
Potential tool for denial of service attacks
Slows down search results on some setups
Has been mistaken as a Google feature
Poor performance on dial-up
Crashes Firefox sometimes
Gives false sense of security
Identifies user as AVG user
Allows simple exploits
Public relations nightmare
Source of ridicule
With careful use of regular (English) expressions all this might be reduced to:
LinkScanner = Nuisance for Good Guys + Bonus for Bad Guys + Danger for Users + Liability for AVG
Submitted for peer review.
Nice list but you forgot about corporate monitors checking employee site access. Poor sod of an employee searches for innocuous site, clicks on site s/he's looking for, gets logged as looking at sites s/he shouldn't be, gets reprimanded / fired / arrested depending on seriousness of site / situation.
"False sense of security" should actually translate to "No security whatsoever" since both good and bad guys serve up "good" pages - or at least serve up AVG pages. :)
I'm not sure what the effect would be of someone searching for private data. Several of my customers use the google search bar instead of the Location/Address field in browsers. This is very common in general. It causes me any amount of hassle sometimes along the lines of "I can't find my new web site / demo site / Control Panel - did you give me the correct address?!" The demo one would be particularly annoying as it would point AVG to my local server. (Same applies to Phorm on that one!)
Also, not just poor performance on dialup - a lot of people still use slow 3-year-old-plus computers (I still log Windows 98 hits) and some (eg me) have slow 256/512 broadband. Even on fast broadband and a reasonably new XP computer my brother complains of the machine slowing down on google listings - or did until I told him to switch off LinkScanner.
After allowing sufficient time for corrections or additions a suitably amended definitive list might be useful to many webmasters, and to our friends at AVG Technologies. Or to The Register.
"mod_security always throws an error for this one"
I really should have remembered that, something similar happened for me.
"Blunders into Good Guys' security traps then marks their site as questionable"
I notice that a simple web search turns up a few sites devoted to LinkScanner.
Some proudly boast that they have had emails from AVG bigwigs Karel Obluk and Roger Thompson seeking help, but none seem to have any real understanding of what is wrong with LinkScanner.
As I told Pat Bitton, WebmasterWorld tells you all you need to know.
Additions for the list:
Reportedly fetches image files by mistake
Reportedly fetches high bandwidth PDF files deliberately
Identifies user on unvisited and potentially incriminating sites
Identifies AVG user IPs in publicly viewable records
But I just noticed that the number of my 'Unidentified' referrals are getting higher again and the number in my "AVG" category are also decreasing.
Some folks here said they changed their UA name from ;1813 to something else, is this true?
If so, how should I filtered them out again since I don't know what's their new name.
Please help. Thanks in advance.