Welcome to WebmasterWorld Guest from 18.204.227.250

Forum Moderators: Ocean10000 & phranque

Message Too Old, No Replies

Thousands of Spambot IPs Hitting my Site Eating up Bandwidth

Thousands of Spambot IPs Hitting my Site

     
2:44 pm on Dec 11, 2011 (gmt 0)

Junior Member

10+ Year Member

joined:Apr 18, 2007
posts:133
votes: 0


Hey guys, need someone's help here. From the past week or so the bandwidth usage of my site increased from 1GB a month to 12GB a Day!

Awstats indicates that there are a range of unique IPs hitting my site and requesting thousands of pages with every visit. Most of these IPs seem to be originating from within the US which is funny. I have blocked China, Brazil and some other countries in my HTaccess but the hits continue.

Here are some example IPs that hit my site as of now and requested thousands of pages.

24.181.178.3
216.6.134.27
70.182.254.242
99.98.188.110
75.134.95.208

Can anyone please explain what is happening? And what can I possibly do to stop this? If this continues, my site will go offline within a week or so.
5:48 pm on Dec 11, 2011 (gmt 0)

Junior Member

10+ Year Member

joined:Apr 18, 2007
posts:133
votes: 0


Please take a look at these ips, these are just a few from thousands that hit my site almost every second. And each one requests around 1000 pages. My site has only around 100 pages so perhaps they request a page over and over again:

65.35.111.110
71.197.69.88
124.123.51.38
173.198.98.134
198.138.135.123
24.159.55.211
50.40.131.171
174.16.100.103
74.131.129.17
75.94.108.222
98.218.136.190
69.244.107.77
68.185.252.101

The funny thing is that all of them look unique and all of them have a verified DNS. How can this be? Could someone please answer.
6:01 pm on Dec 11, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2001
posts:5507
votes: 5


This is better addressed in Search Engine Spider and User Identification Forum [webmasterworld.com]
6:03 pm on Dec 11, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2001
posts:5507
votes: 5


FWIW,
Awstats and these IP's (at least alone) are useless.

You need to view your "raw visitor logs" and see if there is some correlation between the the visitors User Agents and the referring page (s).
7:35 pm on Dec 11, 2011 (gmt 0)

Junior Member

10+ Year Member

joined:Apr 18, 2007
posts:133
votes: 0


Thanks, the referring pages are generally pages within my own site. Have posted it in the search engine spider thread, but yet to get approved.
7:47 pm on Dec 11, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2001
posts:5507
votes: 5


the referring pages are generally pages within my own site


1) You haven't mentioned the User Agents?
Is there any consistency there?
Perhaps obvious bot names or badly formed browser UA's?

2) are the referring pages the same as the requested pages?
8:33 pm on Dec 11, 2011 (gmt 0)

Junior Member

10+ Year Member

joined:Apr 18, 2007
posts:133
votes: 0


I just realized that most requests are direct and have no referrer. The referrer is blank. I used the following in my htaccess as of now and the spam seems to have completely stopped:

RewriteCond %{HTTP:Accept-Language} ^$ [OR]
RewriteCond %{HTTP_REFERER} ^$
RewriteRule .* - [F,L]

I know this will block quite a few legit users as well but is there any other solution to this? Also will this hinder search engine bots from crawling my site?

I also noticed that all these bots are requesting only one page on my site. This page is the largest page my site has. Is it possible to apply this htaccess rule to this single page alone and not the whole site?

Some of the most consistent UAs are as follows:

Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)

Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0; FunWebProducts)

Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0) w:PACBHO60

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; FunWebProducts; GTB7.0; SLCC2; .NET CLR 2.0.50727; .NET CL

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; BTRS28059; SearchToolbar 1.2; GTB7.2; SLCC2; .NET CLR 2.0.50727;

I am not exactly sure if referring pages and requested pages are the same, but I think they are.

PS: the last three UAs are truncated. I am not able to copy the full text for some reason.
2:18 pm on Dec 18, 2011 (gmt 0)

New User

5+ Year Member

joined:Feb 27, 2011
posts:1
votes: 0


In the day job as the internet police, I have seen this behavior in our content filter logs for one specific website run by a local church. When a user visited that website in Internet Explorer, it got stuck in some kind of redirect loop, and requested the same error page almost a half million times in a little under two hours.

When I visit that page in Firefox, I got a "the way the page redirects will never complete" error.

Form where I sit, this looks like some kind of bug in Internet Explorer combined with a broken redirect. Your user example agents are all IE.

I also see "PACBHO60", "FunWebProducts" and "SearchToolbar" in your examples, they seem to be spyware/malware , maybe they are also interfering in how IE redirects.

Al
4:38 am on Dec 19, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 5, 2005
posts: 2065
votes: 2


Al, welcome. This thread sort of duped/looped itself into the Search Engine Spider and User Agent Identification Forum where it continues... [webmasterworld.com...] Here's hoping more info = more ideas:)