homepage Welcome to WebmasterWorld Guest from 54.205.254.108
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Code, Content, and Presentation / Apache Web Server
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL & phranque

Apache Web Server Forum

    
Deny or Allow with RewriteMap
Desire to block apnic cn jp sg etc but allow au nz
privacyman




msg:1522636
 12:43 pm on Nov 21, 2005 (gmt 0)

Whereas IP groups especially APNIC is such a disorganized collage of different countries, a lot of Europe's IP numbers are also intermixed for different countries, blocking outright by an entire group is not possible.

In my situation with shared server, I do have my own .htaccess file which is currently at 900+ lines, including bad bot UA's. I already know that a person can block and then allow such as in the following:

order deny,allow
deny from 202.0.0.0/8
allow from 202.20.103.0/20
allow from 202.40.80.100/24
allow from 202.102.1.
allow from 202.14.200.8

That is an off the wall arbitrary example of allowed IP's. In reality, I would like to block undesired entire groups (example, by the deny 202.0.0.0/8 cidr spec or with deny 202.) but then allow specific countries such as AU and NZ which may have chuncks of IP's in the 202 group. I have often gotten a list of IP's for a country by using the utility at ip.ludost.net to list the IP's by cidr for a specific county.

By using the above means to deny "all" and to then allow only desired IP's by cidr specification (and maybe with a few individual ip's, last item in example) such a list to allow all of AU and NZ in the 202 group but to deny all others would be of considerable length. [Alternatively to block or allow via DNS blocking with country designation could slow down system performance, as well, I have found that means has often "replaced" the IP number in my log file with the domain.name.it
(italy as example) which is not desired.]

Will the RewriteMap function allow specification of allowed (or denied) IP's listed by cidr range or would such an external list have to be written as a LONG list of indiividual IP numbers, example, 202.10.10.1,202.10.10.2,202.10.10.3 and so on as a comma delimited list or one by one on a line.

If RewriteMap "can" accept IP's in the cidr format, then I could simply create my list of allowed IP's by using the utility at ip.ludost.net or by other means, then I might be able to do something like using the RewriteMap to set an env variable then

order deny,allow
deny from 202
deny from 203.
allow from goodguys

Hope someone understands what I am trying to do..... ie, in specific groups to deny most but allow some countries or ip ranges or cidr's WITHOUT having a great big huge listing within the .htaccess file.

Any help on this would be much appreciated. One of my goals would be to block most if not all of Asia from my site yet allow AU and NZ (all of which are in the APNIC group of IP numbers). And if memory serves me correctly I believe I once tried blocking or allowing by country code (DNS lookup) and I did not like it's performance nor (if memory is correct) did I like the fact that my log file got the IP numbers changed to names rather than IP numbers.

thanks, George

 

jdMorgan




msg:1522637
 2:42 pm on Nov 21, 2005 (gmt 0)

RewriteMap is not available in .htaccess context, so are you able to modify httpd.conf?

If so, then you can use RewriteMap to call a script to do the vetting of IP ranges/CIDRs, etc.

If not, then RewriteMap is out, and only two solutions in addition to the ones you've already mentioned come to mind:

First, simply block the IP ranges that cause the most trouble, and live with the few low-runners that get through.

Second, use a dynamic solution, such as the robots.txt-violation detector script [webmasterworld.com], and/or the too-many-fetches-too-fast script [webmasterworld.com], both of which are posted here on WebmasterWorld, and then consolidate and trim your list of banned IPs daily/weekly/monthly as applicable to keep your .htaccess file reasonably short.

Jim

privacyman




msg:1522638
 5:04 pm on Nov 21, 2005 (gmt 0)

Thanks for the reply Jim.

Where my site is hosted (I believe "shared") I don't recall noticing or seeing the httpd.conf file, so I will either use the Cpanel and browse around or I will access my site via msdos's FTP www.sitename.com to see if I can find it and if visible or seen then I can check its permissions. Otherwise I might check with staff at my server to see if it's available to me.

One of the primary reasons that I had for blocking most of apnic ip's (and some other areas) is that (1) my site is not intended for those audiences and that (2) about the only thing that comes from a lot of them is bots (that may or may not be identified) that search for email addresses or that steal code or pages. Odds of getting "legitimate" viewers from jp, cn, tw, hk, sg, etcetera would be alike the odds of winning a lottery, maybe not that high for odds but I would bet probably only 1 out of 500 might be a real person to which the site might be beneficial.

jdMorgan




msg:1522639
 5:34 pm on Nov 21, 2005 (gmt 0)

It doesn't matter why you want to do this -- that's up to you. We talk Apache technology here. I block plenty of IP ranges too, for my own reasons, which is where some of my recommendations come from. ;)

If you are on a shared server, it's highly unlikely that you can modify httpd.conf, since that could affect other users on the same server. So I commend the other approaches to you.

Jim

privacyman




msg:1522640
 6:30 pm on Nov 21, 2005 (gmt 0)

Thanks.

I did check via Cpanel (file manager) to see if I could locate the httpd.conf file, but did not find it. Then I searched the net for "httpd.conf" without quotes which gave one of several results, the one at W3C.org at
[w3.org...]

Whereas the w3c site gave sone info on that page, but they did comment that a server "can" have multiple httpd.conf files. Specifically w3c said "Unlike some other HTTP servers, W3C httpd requires only a single configuration file (but it can have many)."

As a result of reading that info I telephoned my server, tech staff. He said that I would not have access to the "main" httpd.conf on the server, as you said (and he too said) that it would affect ALL domains or accounts. But....

He "did" confirm that I could have my own httpd.conf file and upon asking him where I would place it, I asked if it would go into the main directory same as my regular html files (main web directory folder) alike the location of my .htacess file. He confirmed that it would be the same directory.

I presume that I would set the permissions (chmod) on it so the world could not write or possibly read the file and I might have to include its name in my .htaccess wherein I have <Files .htaccess>deny from all</Files> I may have to include the config file with <Files .htaccess httpd.conf>deny from all</Files> but I am not sure on that part.

Looks like I might have lots more searching of the net to do before I am done with this to figure out exactly how to do it, including to find out if RewriteMap will accept ip numbers listed in a cidr format.

It was, though, very good news finding the page at w3c.org and then getting a confirmation from my server that I could have my own httpd.conf file that would be applicable to my own domain. I will also have to find out if my own httpd.conf file would effectively be a replacement of the root config file (where I would have to specify a whole bunch of stuff) or if mine would be just a "supplement" (or would "add" my own extra items).

If you have any other tips or suggestions that might be helpful would be appreciated, otherwise I think I really have some work cut out for me to achieve this task. A nice learning process.

George

jdMorgan




msg:1522641
 7:08 pm on Nov 21, 2005 (gmt 0)

RewriteMap is just what it sounds like - a map. So, it ascribes no meaning to the data it handles. Therefore, it is not going to accept a CIDR unleess you call it with a CIDR (which would not be a simple project).

Your choices will be either a one-to-one list of single IP addresses, each mapped to a 'good' or 'bad' token , with missing IP addresses returning 'NULL' (this list would likely be unmanageably huge), or you can use RewriteMap to 'call' an external script. This script could analyze %{REMOTE_ADDR} against a CIDR-based list (it's up to you to write the code any way you like), and then return a 'good' or 'bad' token for good/bad IP addresses, or 'NULL' for "not found." RewriteMap then provides this status to the RewriteRule, and you then take appropriate action on it.

Jim

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved