Forum Moderators: open

Message Too Old, No Replies

Microsofts .NET hitting my site like crazy

Need help to stop it

         

DrOliver

2:27 pm on May 24, 2003 (gmt 0)

10+ Year Member



I've read this thread:
[webmasterworld.com...]

But it didn't help me with my problem a lot. I run a small site which gets hit by these so hard, that my site statistics gone useless and my site severely slowed down.

It identifies as:

  • Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1
  • Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1
  • Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1

and comes from:

host217-44-159-86.range217-44.btcentralplus.com

Can someone help stopping this? I have access to .htaccess and robots.txt. I tried to modify .htaccess, but it didn't work out, maybe I did something wrong.

bull

3:08 pm on May 24, 2003 (gmt 0)

10+ Year Member



write in your htaccess:


deny from 217.44.159.86

or


deny from 217.44.159.

to block a larger part

Dreamquick

3:11 pm on May 24, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I doubt this is actually a .NET application as the UA is malformed (no closing bracket), given its behaviour its more likely to be some undesirable bot - you're right to try and ban it.

- Tony

bird

3:27 pm on May 24, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I'm not sure if the real referer is really malformed. Rather looks like DrOlivers stats software truncated the strings. The full thing would look something like this:

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.0.3705)

wilderness

4:23 pm on May 24, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Of course I'm sinister, overbearing and filled with despair!
However BT doesn't bother me any longer.

NTL Internet of GB is also part of BT.

RewriteCond %{REMOTE_ADDR} ^194\.(7[2-5])\. [OR]
RewriteCond %{REMOTE_ADDR} ^195\.47\.36\. [OR]
RewriteCond %{REMOTE_ADDR} ^217\.3([2-9]¦4[0-7])\. [OR]
RewriteCond %{REMOTE_ADDR} ^217\.35\.209\. [OR]
RewriteCond %{REMOTE_ADDR} ^217\.(4[2-4])\. [OR]
RewriteCond %{REMOTE_ADDR} ^62\.7\.(24[0-7])\. [OR]
RewriteCond %{REMOTE_ADDR} ^81\.(12[89]¦1[3-5][0-9])\. [OR]

I have more of this ISP's ranges accumulated. They are just not convereted to strings.
Sticky me if you desire the rest.

DrOliver

9:22 pm on May 24, 2003 (gmt 0)

10+ Year Member



Big thanks to all of you. As it is a small site only, it runs on a cheap server with limited services, but at least they let me have my own .htaccess with full possibilities. Their site stats software is limited too, and I can only check logs once a week, so to see if your advices will help, it will take at least one week. Meanwhile, thanks again and group hug :-)

Dreamquick

2:52 pm on May 27, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Bird,

I've seen the same UA myself and it's malformed as described - it's not just a user typo.

- Tony

bird

3:12 pm on May 27, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



it's not just a user typo.

Ah, never seen that myself, thanks for the info.

In that case, matching for the malformed string is probably more effective than sweeping large IP ranges:

RewriteCond %{HTTP_USER_AGENT} [red][3]CLR\ 1$[/3][/red]

Btw: For those who missed wilderness' joke, I'd recommend to think twice about following his example. I'm don't have exact numbers about btinternets market share, but it looks like a safe bet that he's banning a significant portion of the british online population.

That's fine if you're pronouncedly anti-british, but probably not appropriate for most normal sites... ;)

wilderness

3:58 pm on May 27, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



but it looks like a safe bet that he's banning a significant portion of the british online population.

That's fine if you're pronouncedly anti-british

Bird,
I'm nearly anti everything ;)
At least, those markets which I feel are NOT beneficial to my sites.
Sometime back I was teased about my DE denying as well.

I'm all for the occassional visits from RIPE and other reigons. However when a visitor either promotes a page which I intend to provide reference, it is not my desire to have the masses indexing that linked page as well as major portions of my non-profit site which they can accumulate resources from beyond their conception at that time. Most return later grabbing many pages.
It's a trend I've learned from monitoring my logs.

The content of my sites is primarily North American. It is those markets which I desire. The RIPE interests in specific content does not offer me that capability of monitoring without extending my observations beyond the market range I'm interested in. In other words their visits and links provide more work in an already non-profit situation. As well as (at least in most instances) the required use of online translation.

I guess a simple alternative would be for me to expand my htaccess to specific folders, which I had not thought about previously. At least until composing this reply ;)

Thanks for the help :)

Don

wilderness

2:12 pm on May 28, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



213.78.134.146 - - [28/May/2003:06:50:32 -0700] "GET / HTTP/1.1" 200 9409 "-" "Fetcher/$Revision:_1.2_$ libwww/5.4.0"
213.78.129.0 - 213.78.142.255
netname: ONETEL-BROADBAND-NONNAT-POOL
descr: Onetel UK Broadband Services
descr: Please mail to abuse@onetel.net.uk
country: GB

I rest my case!