Logs Show Surge, but Not Human?

[rogerd]

On one site I work with, I've seen traffic go from 10K visits/day to 40K. The additional traffic looks human at first glance - it is captured by Google Analytics, It comes from diverse consumer IPs in the US and Europe (but not Asia), and the bounce rate is high but one out of ten visits or so loads another page.

On the non-human side, we have all of the traffic coming with no referrer, and it is all focused on a few pages that are hardly viral linkbait and would get one or two views on a good day. It's all IE (spread among 6 - 9), and a range of screen resolutions that look unusually aged (e.g., 1024x768).

Anecdotally, I've heard of a few other sites seeing this kind of traffic, but nobody knows what the purpose might be. It's not scraping, as it's the same pages that get hit. It's not intense enough to be an attack to take the site down, nor is the site likely to be the target of miscreants.

The level of traffic has gone up and down, but it's still happening.

Are any of your sites seeing this, and do you have any theories?

Any thoughts on screening this out of Analytics? It totally blows up time period comparisons.

[netmeg]

One of my sites is seeing this right now, and I'm not crazy about how it's blowing up ad impressions either.

I sent a screenshot off to my host to see if they can make anything of it.

[netmeg]

Strikes me it fits the profile of a windows virus botnet type thing - for me at least, it's all IE (different versions) from all different IP numbers. I dunno what they're looking for - WordPress vulnerabilities? That's what my host thinks, anyway.

I disabled my ads on the site for the time being, as I would rather not be kicked out of AdSense on account of it.

[lucy24]

The additional traffic looks human at first glance - it is captured by Google Analytics, It comes from diverse consumer IPs in the US and Europe (but not Asia), and the bounce rate is high but one out of ten visits or so loads another page.

Is each page request accompanied by requests for all associated files-- css, images, favicon? That's my absolute criterion. Matter of fact I'm still hammering out an alternative ID for mobiles that don't get the favicon. It even works on 403s: humans get the error-page stylesheet, robots don't. Come to think of it, the CSS might be a better criterion for some pages. Except, blast it all, the page that was intentionally made with nothing external. ###

[incrediBILL]

I keep hearing reports of this type of thing over and over from various sources but it seems to have no specific purpose that I can figure out, seems to be meaningless, what's the purpose?

Nobody escalates things like this unless it's an attack, a probe, there must be an underlying strategy.

[rogerd]

Hmmm, netmeg, you mention Wordpress. There is some WP on the site I mentioned, though I'm not sure if the pages in question are part of the Wordpress area.

IncrediBILL, do you know if Wordpress is a common theme? WP is so common, of course, it could be a total coincidence.

[lucy24]

nor is the site likely to be the target of miscreants

Doesn't have to be the site itself. All it takes is one breach and any halfway intelligent robot will be running wild all over the server-- and beyond. If you're looking for vulnerabilities, look for the WordPress site hosted by a 13-year-old who gets two human visits a month. That's why you get all those random robots snuffling around for nonexistent php files. Keep trying doors and you'll find one that's unlocked.

[rogerd]

Good point, Lucy, I've seen plenty of hack attempts that just look for a software footprint and start probing. The volume seems odd, though - why keep hitting the same pages on the same site tens of thousands of times over a period of days?

[netmeg]

At this rate, I'm going to have to take my site down. It's not a hobby, it's a business, and I can't monetize it with fake traffic. Geezopete.

[scooterdude]

It may take a while , but ultimately advertisers will begin to demand the same level of traffic verification/authentication as they get with print media with certified circulation

some have being watching and hearing of fake traffic rise from low levels and usually only on low level sites, now , perhaps biggger fish get to enjoy this largesse

folk wonder why their traffic no longer converts, and then continue to measure their remaining conversion using the same 'convinient' tool they've always used and believed in, could it be that they still geting the same traffic , sans the small percentage that actually converts, which they've lovingly identified and quantified over years, but you say, surely only the web master is privy to such data :)


Oh yes, back on topic, What analytics package is used on said site, and how long has it being in situ, if you can say off course ?

[netmeg]

Google Analytics, Statcounter, Woopra. All reporting the same.

[rogerd]

GA is logging the traffic as normal, so the bots are loading scripts.

[ThatsBoBo]

netmeg: How do you disable your Adsense ads on your site wehn under attack? Do you simply remove your domain from the "allowed sites"?

Thanks.

[rogerd]

On a dynamic site (Wordpress, Drupal, etc.) it's fairly trivial to delete the tags or substitute them for house ads if the formatting would look odd, ThatsBoBo. If you are serving your own ads from OpenX or similar, it would be even easier to swap out the Adsense ads for something else.

[netmeg]

My ads are placed via my customized functions file; I just commented them out.

[Filipe]

How funny that this appears as a featured discussion, we're seeing the exact same behavior. Here's basically what we're seeing:

• About 500 new visits every hour
  
• User agent is usually some version of Internet Explorer
  
• IPs and ISPS are not consistent, usually coming from the Americas / Europe
  
• AFAIK: 100% bounce rate
  
• AFAIK: Time on site is under 1 second
  
• Hitting at least Google Analytics (potentially hitting all resources listed on homepage)
  
• Not generating significant load on the server, I don't think it's crawling
  
• Traffic from these sources has been consistent since about 14:00 PST (UTC -8) yesterday
  

It's the strangest thing. If it's a DDOS thing, it's a pretty poorly executed one. Hopefully they're not just testing our defenses :|

[netmeg]

How many URLs on your site are they hitting?

[Web_speed]

Could be email scrapers. These are win vb programs and they use the IE browser as a plug in. The visitor will be logged as a normal IE browser and the page (including java script/images/css) will be executed like in any IE browser.

I guess the only way to detect and separate them is to look for mouse movements as well as doing some other human/bot tests.

I've noticed a major increase in email spam across my domains lately. This may be a direct result of the above mentioned.

[hottrout]

I have to report the exact same problem with my site. At approx 18:00 GMT yesterday (21st Feb 2012) my root url / jumped from a daily average of 500 direct hits (no referrer) to 16,000 today (22nd). It does not crawl but the count is registered in GA. At the same time and as a result the bounce rate has jumped to 70% and time on site average has dropped. The hit seems to take 1s. It is coming from North America but spread over all regions.

It does not feel like any DOS I have seen but it is very weird. My forum (phpbb3) is also registering the stats as guest members which I find odd.

Does anyone have any clue as to what is happening?

[rogerd]

If they were hunting for emails, I'd expect them to scour thousands of pages, rather than hitting the same three pages thousands of times.

I did read, Web_speed, that one or two large email spam botnets have been resurrected very recently.

[honestman]

Certainly an exploitation of IE (or some plug-ins) according to my stats. Not sure how to stop this technically. Any specific ideas (thank you Web_speed for pointing to the issue)? I don't want to make entering web pages on the site onerous to the user.

[breeks]

Same here. All the traffic is going to only two URLs. Traffic is from North America but spread over all regions.

[seoskunk]

I have posted twice about this zombie traffic. In my case it sent bounce rates through the roof and visitor stays through the floor so if thats a factor in rankings its a type of google bowl at the site (but probably I'm paranoid). I did post this when I saw traffic was zombie.

To try and prevent this I put in a spider trap and my traffic dropped and bounce rate went from 70% to 38% and time on site also increased longevity of visitors.

Previous posts
[webmasterworld.com...]
[webmasterworld.com...]

[Dinkar]

It may be (fake) referral link spam.

[netmeg]

I don't know what that means. It's not spam, and there's no referrer.

[Seb7]

You can send a message to all the users of these bot machines just by using a javascript alert box.

I suggest adding a honey pot for the bot to navigate to (a hidden link to another page), where the hidden page simply displays a javascript alert saying 'hey, your machine seems to be infected with a virus'.

Though IE can be controlled in a hidden window, any alert boxes still get shown on the desktop.

[tangor]

I suspect that these are not desktop hits... the smartphone crowd (coders) are working over time... and might be time for some who have not already embraced the "dark side" of banning no referer or no UA to join the club.

Whatever choice, it is becoming more difficult week by week to know which hits are HUMANS and which are BOTS and which are PRE-FETCH to make the phone work better...

[g1smd]

If there's some sort of commonality in the vast majority of unwanted hits a few lines of mod_rewrite forcing [F] or [G] might just do the trick.

[enigma1]

There are ways around it. You could output the analytics code after a second page is accessed based on IP. This should filter pretty much most of it. Of course it will detriment the value of bounce rates too in ga but you could extract it from the logs.

I don't know if the recent changes in the google SERPs have anything to do with it, if they amplify it. In some cases now, I cannot get straight away into a site clicking the found links and have to enter them manually mainly because of the extensions I use with the browser although I had no trouble couple of weeks ago. Maybe there are similar problems with portable devices but I haven't run any tests yet.

[netmeg]

If there's any commonality (other than no referrer) I and my host haven't found it yet.

Still going.