homepage Welcome to WebmasterWorld Guest from 107.21.163.227
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Accredited PayPal World Seller

Visit PubCon.com
Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
Forum Library, Charter, Moderators: Receptional & mademetop

Website Analytics - Tracking and Logging Forum

This 354 message thread spans 12 pages: < < 354 ( 1 2 3 4 5 6 7 [8] 9 10 11 12 > >     
Logs Show Surge, but Not Human?
rogerd




msg:4420176
 9:23 pm on Feb 21, 2012 (gmt 0)

On one site I work with, I've seen traffic go from 10K visits/day to 40K. The additional traffic looks human at first glance - it is captured by Google Analytics, It comes from diverse consumer IPs in the US and Europe (but not Asia), and the bounce rate is high but one out of ten visits or so loads another page.

On the non-human side, we have all of the traffic coming with no referrer, and it is all focused on a few pages that are hardly viral linkbait and would get one or two views on a good day. It's all IE (spread among 6 - 9), and a range of screen resolutions that look unusually aged (e.g., 1024x768).

Anecdotally, I've heard of a few other sites seeing this kind of traffic, but nobody knows what the purpose might be. It's not scraping, as it's the same pages that get hit. It's not intense enough to be an attack to take the site down, nor is the site likely to be the target of miscreants.

The level of traffic has gone up and down, but it's still happening.

Are any of your sites seeing this, and do you have any theories?

Any thoughts on screening this out of Analytics? It totally blows up time period comparisons.

 

macavity




msg:4433685
 9:22 pm on Mar 26, 2012 (gmt 0)

I guess it's not too surprising (given their subscriber numbers) but it's useful to know that road runner and comcast are topping a number of people's lists. I'm going to start with these two and see what response I get from their abuse contacts.

Future




msg:4433689
 9:26 pm on Mar 26, 2012 (gmt 0)

thanks for raising the topic,
we have a huge list of traffic from rr, on few sites !

is there anyway to track these ip, when i am not using analytics for few sites ?

this data should be helpful for many here

rogerd




msg:4433695
 9:31 pm on Mar 26, 2012 (gmt 0)

Your logs will show the IPs, Future. Since these appear to be consumer IPs, though, they do not persist over time. Earlier in the thread one member reported some success with banning the IPs, but over a period of days the traffic returned from new ones.

netmeg




msg:4433913
 2:11 pm on Mar 27, 2012 (gmt 0)

My traffic has been very slowly inching up again.

RoadRunner, Comcast and Verizon are the top ISPs.

I don't really have a way to track which IP numbers match up to which ISP in order to send them a list, though.

On the up side, I wrote a blog post about it and now I rank on the first page in Google for "zombie robots." (!)

Oh, I also tried Cloudflare for a bit, but it only identified around 300 or so possible botnet hits out of thousands and thousands. Those were probly the ones I *normally* get.

staffjam




msg:4434219
 1:35 am on Mar 28, 2012 (gmt 0)

It's really picked up again for me. It had dropped to about 10,000 a day and now it's close to 30,000 a day. This thing isn't going away.

Edge




msg:4434589
 10:10 pm on Mar 28, 2012 (gmt 0)

FYI, most of my bot traffic indicated as Windows 7.

(Corrected from Vista_ ...Edge

Mario155




msg:4435221
 12:51 pm on Mar 30, 2012 (gmt 0)

what the heck, this is coming back again, my traffic has increase by thousands again. The page they are going to is my main home page, so I cant delete it.

One thing I noticed is that a most of the bots from this attack are not using the www for my site, meaning they aren't going to www.mysite.com, they are going to mysite.com

netmeg




msg:4435285
 3:10 pm on Mar 30, 2012 (gmt 0)

Yea, mine have starting gaining again over the past couple days.

My www 301's to my non-www.

Mario155




msg:4435297
 3:53 pm on Mar 30, 2012 (gmt 0)

What are we gonna do about this? We have to contact someone. I see some people have stated they are getting 30,000 extra visitors. I am not getting that much yet, but if I do I don't want it to slow down my site.

What if this never stops? What if it just keeps going more and more until the sites are nothing but bots, and there are so many bots that the sites all crash?

netmeg




msg:4435448
 11:07 pm on Mar 30, 2012 (gmt 0)

I got nothing.

Edge




msg:4435558
 1:12 pm on Mar 31, 2012 (gmt 0)

I have verified a suspect application - ever heard of "Graphite Browser"?

Don't ask, I won't tell why...

Mario155




msg:4435564
 1:33 pm on Mar 31, 2012 (gmt 0)

Graphite Browser brings up IE9 and Windows 7 in Google, and those are the 2 applications that these bots appear to be using.

I don't know if this has been asked yet, but has anyone stated what they use on their site? Maybe there is a common link as in vBulletin forum, or Wordpress, etc.

The site of mine that is getting attacked uses Drupal, and also has a vBulletin forum. The bots are not going to the forum though. Do you guys use a CMS? Or just HTML?

Edge




msg:4435573
 2:07 pm on Mar 31, 2012 (gmt 0)

Mario155 - you're looking for a motive so am I however, I would rather just block and rid of this botnet completely and forever.

From what I have seen of the particular installations they seem to be interested in tracking several ad networks. AdSense did not appear to be in the configuration.

Because of how this bot Browser is configured, I suspect there are many “collateral damage” websites out there. This is a keyword thing…( I think).

Though in view of how this botnet appears to be growing in scale, it could be a malicious attempt of some flavor against publishers and/or advertisers.

It is likely that my keyword competitors are seeing this botnet traffic as well.

netmeg




msg:4435581
 2:52 pm on Mar 31, 2012 (gmt 0)

Mario you should probably go back and read over the entire post. There's no common link. My site is WordPress, other sites are static HTML or other CMS packages. Some sites have ads, some do not. I don't happen to think this one has anything to do with keywords given the nature of my site that's being hit. I do think it's some kind of collateral damage (i.e. they're not hitting me or my site intentionally) because if someone wanted to hit me, there are better ways to do it.

netmeg




msg:4436660
 1:57 pm on Apr 3, 2012 (gmt 0)

Don't get your hopes up, but Google did reach out to me for more information and have been all over my blog post on the subject. I doubt I'll hear back anything specific, but they're definitely aware of it. I don't know if there's anything they can do about it, but they have more resources than I do.

macavity




msg:4436827
 7:27 pm on Apr 3, 2012 (gmt 0)

Nice one netmeg, hope something comes of it.

ken_b




msg:4437192
 7:25 pm on Apr 4, 2012 (gmt 0)

This is pretty annoying. Really screws up stats.

I finally took Google Analytics off my home page (only) and replaced it with Statcounter, just for the home page.

.

hottrout




msg:4437578
 4:47 pm on Apr 5, 2012 (gmt 0)

I have some additional information to add to this that may or may not be related.

I have noticed in my webserver stats that I am receiving very high hits from www.cj.com. In fact since last month they are the second largest provider of requests after my own URL. I do not use cj.com and was wondering if this was related. Could it also be that the bot/script is masking itsself as cj.com?

I have also dropped on one of my main seaches in google from a 7 year No1 position to a no5 position?

ken_b




msg:4438527
 4:49 pm on Apr 8, 2012 (gmt 0)

Since I changed from Google Analytics to Statcounter for my homepage I noticed via SC that almost all of my homepage visits were entering the site via example.com, not www.example.com.

at first I didn't get what this meant. But the other day I clicked on the example.com link in the SC entry page list and it took me to my site as example.com. It should have redirected to www.example.com.

Somehow the redirect in my htaccess had stopped working after several years of working just fine.

No idea how that happened.

But I had noticed when I visit WMT that G was showing me as owning two sites, example.com and www.example.com. I didn't think much about that, but if I recall right, that started showing up about the same time as the rogue traffic.

No idea if there is any connection, but I thought I'd mention it.

(I reloaded my htaccess file and it's working fine again)
.

eviljim




msg:4439420
 3:22 am on Apr 11, 2012 (gmt 0)

I'm also getting hit by this (have been since the 21st). It appears to be focused only on my index page.

- vast majority shows as IE9, but there is some 8 and 7.
- peaked on the 21st then slowed down a bit, but has hovered at around 2500 hits per day
- loads javascript (shows up in analytics, loads up my openx ad).
- impressions don't show in google adsense
- but, I strongly suspect they're affecting google adsense in some capacity; my eCPMs and income *significantly*, undeniable drop since the 21st. [By approximately 75%].
- this sucks

I'm attempting to block traffic with javascript; hopefully that will prevent this robotraffico from loading adsense/analytics.

Wouldn't that be nice.

hottrout




msg:4439481
 8:37 am on Apr 11, 2012 (gmt 0)

Has anyone else got any ideas on this. My stats are still out and the levels of zombie hits are constant. There has to be some way to stop or prevent these hits?

ken_b




msg:4439591
 1:32 pm on Apr 11, 2012 (gmt 0)

eviljim
It appears to be focused only on my index page.

If you have AdSense on the indexpage I'd sure recommend taking taking it off that page, or any page getting hit by this thing.
.

Mario155




msg:4439614
 2:41 pm on Apr 11, 2012 (gmt 0)

I know, I need to get this stopped too. It has been going on for nearly 2 months. My peak season is coming soon and I cant have these bots crashing my site when my traffic increases for the summer.

From what I can see, this is starting to increase again. I am getting at least 10,000 hits a day from these.

hottrout




msg:4439621
 2:54 pm on Apr 11, 2012 (gmt 0)

My average stats look like this.

AVG visit per day before 21st Feb 8000-9000

Visits peeked at 27,000 on the 22nd Feb

These figures then gradually droped until they reached their new average of 13,000 to 14,000 visit per day from the 1st March onwards.

This means that I am experiencing an average of 5000 to 6000 zombie visits per day ever since.

I have now got my server providers involved and asked that they live monitor and investigate the matter as noone else seems to have an answer.

Key_Master




msg:4439651
 3:45 pm on Apr 11, 2012 (gmt 0)

Some years ago, AVG's LinkScanner component was causing similar amounts of traffic spikes from IE users. It's possible these hits are coming from some kind of prefetch virus scan. Unfortunately, nobody has provided any tangible information about these hits. No HTTP headers or raw data provided by a good JavaScript sniffer have been posted.

If my sites were overwhelmed by this traffic and the data I gathered wasn't sufficient enough to find a solution, I would isolate these visits and conduct a distributed port scan on each IP. At 25 scans per IP, it would only require slightly over 2600 unique visitors to conduct a full scan of each of the 65535 ports. I'd then take the list of open ports I collected and rerun the experiment to see if these zombie visitors shared a common port. My guess is, a port would be discovered that would lead to an answer about these hits, and ultimately, how to defeat them.

netmeg




msg:4439705
 6:24 pm on Apr 11, 2012 (gmt 0)

We did collect the headers, and and sniffed the raw data (but I'm not posting anything here, thanks) We were unable to determine anything we could use to block them.

My bots are inching back up as well.

Edge




msg:4440035
 2:24 pm on Apr 12, 2012 (gmt 0)

I have been able to control the botnet visits to a manageable level. I'm knocking out approximately 90% + of the hits – millions of monthly page views…

This is done via. htaccess language and ip blocking. About every four-five days I update my ip ban list to the top 3800 offenders. I have actually banned entire ip ranges of regions/networks that hit often and are of little to no monetary value to my vertical.

AdSense revenue is/has recovered and life moves on.

macavity




msg:4440048
 2:56 pm on Apr 12, 2012 (gmt 0)

Edge,

Are you able to share any insight into how you're identifying the IPs in question? Are you trawling your server logs for a specific string? (I confess I haven't yet had the time to look into our logs in detail).

Edge




msg:4440059
 3:34 pm on Apr 12, 2012 (gmt 0)

macavity,

I have four webpages that are getting hit. Three normally have low to high traffic; one of these webpages normally has almost no traffic.

I put "AXS Visitor Tracking System" script (Google it) on that “no traffic” webpage and let it run.

After about 24 hours I have quite a collection of ip addresses. I copy the ip addresses and paste into an excel file. Then I trim and edit the ip addresses list down to an htaccess manageable level. I trim the one to six or more hit wonders and keep the repeat ip address offenders.

I then insert “deny from” in a column ahead of the ip addresses within my excel file, then copy and paste into a notepad document.

Then, I copy and paste the notepad document into my htaccess file (Notepad cleans the text so to speak).

Repeat every three to seven days as required…

As I identify certain regions/langauges of the world as repeat and non-value offenders – I block that region via language block in my htaccess; e.g., Portugal (pt), Brazil, Mexico (es) and Netherlands (nl). I slao block certain ip ranges based on excessive hits from the range.

I actually have another idea I plan to implement is the next couple of weeks.

macavity




msg:4440063
 3:48 pm on Apr 12, 2012 (gmt 0)

Thanks Edge. We're currently only seeing this on a single homepage so it's (I guess) not so easy to distinguish good traffic from bad. Nevertheless, it's useful to know about your system. Thanks for sharing.

eviljim




msg:4440086
 4:32 pm on Apr 12, 2012 (gmt 0)

So you do see the same IPs as return visitors at some point? I'd not done a huge analysis but the IPs always seemed unique.

Interesting.

This 354 message thread spans 12 pages: < < 354 ( 1 2 3 4 5 6 7 [8] 9 10 11 12 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved