homepage Welcome to WebmasterWorld Guest from 54.211.157.103
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

This 36 message thread spans 2 pages: < < 36 ( 1 [2]     
More Chinanet
119.128.0.0
not2easy




msg:4521841
 10:24 am on Nov 23, 2012 (gmt 0)

119.130.19.83 - - [20/Nov/2012:20:16:03 -0600] "GET / HTTP/1.1" 200 9816 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-us; rv:1.9.2.3) Gecko/20100401 YFF35 Firefox/3.6.3"
119.130.19.83

119.128.0.0 - 119.143.255.255
More Chinanet Telecom services (Guangdong)

Robots - NO
It tried another 50+ 403's after it was blocked. Requests included the nonexistant : 119.130.19.83 - - [20/Nov/2012:20:27:33 -0600] "GET /self.location HTTP/1.1"

 

lucy24




msg:4522427
 9:49 pm on Nov 25, 2012 (gmt 0)

deny from 49.50.4.0/22 49.50.8.0/22 110.136.176.0/20 110.139.0.0/16 114.79.18.0/24

Well, ###. Yet another thing I didn't realize you could do.

Does this format run faster than one deny per line, as in
deny from 49.50.4.0/22
deny from 49.50.8.0/22
deny from 110.136.176.0/20
deny from 110.139.0.0/16
deny from 114.79.18.0/24

?

Incidentally, I've got 110.136.0.0/14 as a single block. Did I miss a bit of Australia hiding in the middle? Also 114.79.0.0/18 as a country unit (not necessarily all the same ISP). But I don't pay much attention to Indonesia.

keyplyr




msg:4522443
 12:13 am on Nov 26, 2012 (gmt 0)

No, using a command for every IP is not as fast as grouping many IPs on a line for 2 reasons:

1.) takes extra bytes for the "deny from" code.

2.) each line is read per command. Starting a new command for each IP is more processing.

I start a new line for each A class just as a management technique, CIDRs separated by a single space. Example:

deny from 1.80.0.0/13 1.202.0.0/15
deny from 2.96.0.0/16 2.97.0.0/32 2.103.150.0/23
deny from 5.9.0.0/16
deny from 8.28.16.0/23
deny from 14.208.0.0/12
deny from 20.0.0.0/8
deny from 23.19.0.0/16 23.20.0.0/14
deny from 31.170.160.0/22 31.184.238.0/24 31.214.145.0/24 31.214.201.0/24
deny from 37.9.53.0/24 37.9.61.0/24 37.16.72.0/24 37.17.0.0/17 37.59.0.0/18 37.59.82.48/28 37.72.170.96/29 37.220.0.0/19

not2easy




msg:4522455
 1:39 am on Nov 26, 2012 (gmt 0)

Thank you so much for clarifying,, keyplyr, now I know where I'm heading with the country codes. I had never entered multiple CIDRs in a single line before, didn't know we could do that. It would be dizzying to edit lines that are too long, so I'm adding them in related batches. Unless someday I find out that numerical order is important for performance.

lucy24




msg:4522469
 2:55 am on Nov 26, 2012 (gmt 0)

:: detour to horse's mouth [httpd.apache.org] to read up on mod_authz_host and see if there's any more stuff that everyone in the world except me has always known ::

Ken_S




msg:4522577
 1:37 pm on Nov 26, 2012 (gmt 0)

Is this considered part of the Chinanet group?

<> 115.236.51.176 - 115.236.51.191

115.236.51.xxx - - [25/Nov/2012:23:47:14 -0800] "GET / HTTP/1.1" 403 3251 "www.google.com" "Explore 6.0"

Ken

keyplyr




msg:4522650
 7:29 pm on Nov 26, 2012 (gmt 0)

@ Ken_S IMO yes, it looks to be maintained by Chinanet.

Just a FYI - The link to China ranges (msg:4521970) is old and there are many important missing ranges.

This 36 message thread spans 2 pages: < < 36 ( 1 [2]
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved