homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
Forum Library, Charter, Moderators: Receptional & mademetop

Website Analytics - Tracking and Logging Forum

This 354 message thread spans 12 pages: < < 354 ( 1 2 [3] 4 5 6 7 8 9 10 11 ... 12 > >     
Logs Show Surge, but Not Human?

 9:23 pm on Feb 21, 2012 (gmt 0)

On one site I work with, I've seen traffic go from 10K visits/day to 40K. The additional traffic looks human at first glance - it is captured by Google Analytics, It comes from diverse consumer IPs in the US and Europe (but not Asia), and the bounce rate is high but one out of ten visits or so loads another page.

On the non-human side, we have all of the traffic coming with no referrer, and it is all focused on a few pages that are hardly viral linkbait and would get one or two views on a good day. It's all IE (spread among 6 - 9), and a range of screen resolutions that look unusually aged (e.g., 1024x768).

Anecdotally, I've heard of a few other sites seeing this kind of traffic, but nobody knows what the purpose might be. It's not scraping, as it's the same pages that get hit. It's not intense enough to be an attack to take the site down, nor is the site likely to be the target of miscreants.

The level of traffic has gone up and down, but it's still happening.

Are any of your sites seeing this, and do you have any theories?

Any thoughts on screening this out of Analytics? It totally blows up time period comparisons.



 6:18 pm on Feb 26, 2012 (gmt 0)

Bummer. Still getting about the same amount of traffic as before. It appears that dracula might be hitting my site from Transylvania :)


 10:05 pm on Feb 27, 2012 (gmt 0)

I was wondering if anyone has had any further luck with this? It's been going on for 2 weeks with my site - i'm averaging 20,000 visits a day and it's throwing my stats out. My hosting provider and webmaster can't figure it out. one lead i did find is: [groups.google.com...]

Any updates or advice would be greatly appreciated.


 12:07 am on Feb 28, 2012 (gmt 0)

its only affecting one page so took off analytics code and adsense on that page. Added some cpm banners which are doing quite well.


 12:13 am on Feb 28, 2012 (gmt 0)

took off analytics code and adsense

I understand taking off AdSense code, but why take off analytics?

For me the strange traffic is continuing to drop off, but I just wish it would go awat faster.

I took off AdSense on that page, no big loss.


 1:45 am on Feb 28, 2012 (gmt 0)

I haven't taken analytics off yet, but I'm thinking about it. It's screwing up my stats, and my analytics are what I need to sell direct advertising. Not to mention measure performance from one year to the next.

(I couldn't make that link to the google group work; it just takes me to whole google group overview)

In my case, it's hitting the home page, which is the top earning page. So it's a big deal. And there doesn't seem to be a way to block it without blocking direct users or search engine bots. At least, no way that wouldn't get me booted out of Google for cloaking.


 1:55 am on Feb 28, 2012 (gmt 0)

I got through to the Google Groups link, and did a little more research. It seems possible that this activity is generated by the Gomez Peer Zone, a sort of voluntary, paid botnet operated by Compuware. It can be used for load testing purposes, as the firm offers that service as well as monitoring. No hard info yet, but the Gomez thing is, in fact, Windows only.


 1:58 am on Feb 28, 2012 (gmt 0)

one lead i did find is: [groups.google.com...]

... Click, read boilerplate about New and Old Groups, sign in, read boilerplate about Old Groups-- of which I have none that I know of-- return to Forums, click, lather, rinse, repeat ... and finally Right-click, Copy Link, paste into browser's address bar, delete WebmasterWorld preface and see what you're talking about ;)

On the micro scale, it looks as if the robots have just about lost all interest in Alonzo & Melissa since I put the file on a crash diet-- less than 5% of its previous weight-- and hid the rest of it behind the door. Lazy little buggers, aren't they?


 2:18 am on Feb 28, 2012 (gmt 0)

Copy Link, paste into browser's address bar, delete WebmasterWorld preface and see what you're talking about ;)

Yeah that works, thanks!


 2:44 am on Feb 28, 2012 (gmt 0)

Seeing as a few people are having trouble with the link i thought i'd post the message from Google Groups - hope this is okay:


This isn't actually a question, but actually a warning. I looked for an answer to my problem here a couple of days ago, couldn't find an answer, did a lot of had scratching and eventually found the answer which I'd like to share with you.

On Wednesday at about 4pm I noticed on Analytics a surge in direct traffic for some reason. I wasn't unduly worried. I then checked on my iPhone App later in the evening, and noticed overall our site bounce rate had gone up to 70%. I just thought I was being attacked by a couple of static IP's from somewhere (why do they do that I wonder). Annoying as our bounce rate is around 30% normally. We're a niche site and only get around 300 visits per day, so a few bounces can make a difference.

Anyway, Thursday morning I came in and checked Analytics proper, and we'd had 900 visits between midnight and 9am, with 99.8% bounce rate. We had slipped down the rankings by quite a few notches as well which was annoying. So we checked out the domains, and there were all pretty usual ISP's, so we checked the IP addresses, and they were pretty much all dynamic. But something very odd was happening and we were getting quite concerned, as when every hour passed we were getting hundreds of bounces an hour.

Then, I received this e-mail:

"Hi Tim,
Having spoken with Chris yesterday, I believe you are the best person to speak with about your website.
I performed a quick test on your home page last night which is attached below. As you can see the availability was around 95%, equating to 5% of customers being unable to access the site.
End-to-End Chart: Average
Test name:
True Traveller FF Agent
Time Period:
Tue, 27 Sep 2011 11:00 to Wed, 28 Sep 2011 11:00
(GMT) Greenwich Mean Time : Dublin, London

graph attached etc

I work for Compuware Gomez who are the Market Leaders in Monitoring Web Applications and work with over 3000+ companies around the world, including 12 of the top 20 most visited websites, to optimise customer experience and increase conversion rates.

Here is a brief overview of what the Gomez platform has to offer:
Monitor the real-time performance of your website from the customer perspective
Be notified instantly of any outages or slowdowns that impact your customers and revenue
Have detailed diagnostics information at hand to minimize troubleshooting and fire fighting
Test to ensure your applications can handle expected traffic load/capacity
Validate visual functionality across multiple browsers, operating systems and screen resolutions
Feel free to get in touch for further information or a demo of our services tailored to you.

Kind Regards,
Lois Aldington
Business Development
Compuware Limited"

So I looked up Gomez, and saw that these "homeworkers" allow a program to be installed on their home PC, and Gomez call it up to "check" sites. These homeworkers are paid $5 per month or something. They have 150,000 distributed nodes!

So I had an e-mail battle with Compuware, and they said "These hits on your website are nothing to with us... we'll stop monitoring it" etc. Suddenly, all these hits to our homepage stopped. Our IT people said this was probably a breach of the Computer Misuse Act 1990.

So the moral of this is if you suddenly have a massive surge of direct traffic to your homepage all with 100% bounce rates from a variety of ISP's from all over the world, check with your guys to ensure no one from Compuware has phoned up trying to sell you something!


 3:15 am on Feb 28, 2012 (gmt 0)

Please provide me with the email address so I can complain as well. Sounds like the exact thing for me. Are these the guys? [compuware.com...]


 3:23 am on Feb 28, 2012 (gmt 0)

I didn't get any emails. And if that was the case, why would they leave the stupid thing running indefinitely?


 7:51 am on Feb 28, 2012 (gmt 0)

I think everyone should get their home pages to also load Compuware home page by using a small iframe or something, just so they get a fair share of their own hits!


 12:36 pm on Feb 28, 2012 (gmt 0)

Note that while this looks suspicious, we don't have actual confirmation that Compuware/Gomez is the source of this traffic.


 1:21 pm on Feb 28, 2012 (gmt 0)



 3:53 pm on Feb 28, 2012 (gmt 0)

After peaking at 15,000 zombies one day they seem to be slowly dying off. Looking at my stats they only seem to come out at night. When the sun rises zombie traffic falls.


 8:29 pm on Feb 28, 2012 (gmt 0)

DOH! Ignore previous code! I just realised the easiest way round this is just to ignore no referrer visits in Analytics, and get stats back on track and you can do this in javascript. I changed analytic code to this, no more problems and my stats read normal again.

<script type="text/javascript">
if (document.referrer != '') {
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'YOURACCOUNT']);

(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);


 12:05 am on Feb 29, 2012 (gmt 0)

It's now increasing for me - about 22,000 per day now. Thanks for the above code seoskunk - sadly mine doesn't have no referrer - it's just direct traffic (i used to receive about 8k direct visitors a day - so can't block all of it.)

Really not sure where to go from here.
Anyone else have any ideas?


 12:17 am on Feb 29, 2012 (gmt 0)

you can do this in javascript

Do we have stats on how many of the growing number of humans with js disabled/NoScript browers will fail?

Some years back I 403'd blank UA's... I'm a half inch way from doing same for Referers (sic)


 1:39 am on Feb 29, 2012 (gmt 0)

Some years back I 403'd blank UA's... I'm a half inch way from doing same for Referers (sic)

Compromise: block everything but your front page. Otherwise you're slamming the door on normal humans who started on some interior page and want to see what the overall site is about. It doesn't take a whole lot of know-how to manually delete everything in your browser's address bar after the domain name, and hit Return.

You could set a cookie saying they've been here before, but that's just a variation on the identical problem.


 1:34 pm on Feb 29, 2012 (gmt 0)

Unfortunately, mine's hitting the home page only.


 2:09 pm on Feb 29, 2012 (gmt 0)

Yep, a portion of the weird traffic is hitting the home page, which makes it next to impossible to block. The individual pages could be moved, renamed, etc. without serious loss, but not the home page.

Unfortunately, it's on the rise again.


 2:10 pm on Feb 29, 2012 (gmt 0)

Considering that the ips may be getting recycled seeing as there is such high volume, i suppose some of you with high distortions have considered recording the IPs and UAs and perhaps sorting out a few days worth to see which ones might jump out as consistent


 2:19 pm on Feb 29, 2012 (gmt 0)

I just called the guys at Gomez and i must say they are very nice. They are opening a ticket and looking as to what the problem could be - whether it's them or something else causing the problems. I'll report back to you guys and let you know what the findings are (they say they should have a report completed today.)


 3:48 pm on Feb 29, 2012 (gmt 0)

This strange traffic has drpped about 75% on my site, but seems to have almost leveled off at that level the last couple of days.

Anyone have an idea of how widespread this problem is? Doesn't seem like that many members here are affected judging by the number of posters in this thread.


 4:20 pm on Feb 29, 2012 (gmt 0)

Can anybody provide a complete list of http headers/values these visitors are sending?


 5:08 pm on Feb 29, 2012 (gmt 0)

I'm looking at a two-pronged approach to this:
1) I wrote a blog post about the problem hoping to find more fellow victims and also get a response from Compuware/Gomez. (I called their attention to the post just in case their social monitoring tools aren't up to snuff...)
2) If there is no identifiable signature, I plan to collect a mass of data and file abuse complaints with the top 5 or 10 ISPs sending the bogus traffic. They should be able to track this back to individual users and either alert them to a malware infection or determine if Gomez is doing this. I expect this will be a slow process, but one cooperative ISP could crack this wide open.


 8:41 pm on Feb 29, 2012 (gmt 0)

I got a mess o' logfiles I could contribute.


 9:47 pm on Feb 29, 2012 (gmt 0)

I would need the full range of http headers this agent is using (e.g., HTTP_CONNECTION).

Because of the low screen resolutions reported, it's possible that this agent is a screen saver utility (trojan?) of some type. Having the full set of headers would go a long way towards solving this mystery.


 11:15 pm on Feb 29, 2012 (gmt 0)

Just heard back fromk Gomez - our site is not currently in their database so that rules Gomez out. Still they have offered to investigate the situation for me - so will keep you posted as to their findings.


 3:00 am on Mar 1, 2012 (gmt 0)

My sites are seasonal, and I can't keep this one up the way it is, and there doesn't appear to be any way to track it or stop this traffic. I am going to have to find another domain and start over. I can't even redirect the old one, because it will just bring the traffic with it. Which is too bad, because it ranks pretty well. Easy come easy go, I guess.


 7:36 am on Mar 1, 2012 (gmt 0)

Does the bot hold cookies?

This 354 message thread spans 12 pages: < < 354 ( 1 2 [3] 4 5 6 7 8 9 10 11 ... 12 > >
Global Options:
 top home search open messages active posts  

Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved