Forum Moderators: phranque

Message Too Old, No Replies

url blocking with access.conf or .htaccess

         

bbxrider

2:40 am on Nov 11, 2010 (gmt 0)

10+ Year Member



I have an apache development server with 6ish sites,
want to block url's by country
the way i'm reading it, it appears I can put an access.conf in my htdocs directory with some thing like this
<Directory /some/path/htdocs/>
order allow, deny
allow from all
deny from 10. 20. 30. etc etc
</Directory>
this will block access to sites on my server from country's (or just url's) whose urls's start with 10, 20, or 30
yes? or can do same as above only in a .htaccess in the htdocs directory?
with 6ish sites I don't want to repeat the blocks for each site, with .htaccess files in each home directory. I want to establish for the entire server
if this is possible, do I need something specific in my httpd.conf file to enable this approach?

bbxrider

3:45 am on Nov 12, 2010 (gmt 0)

10+ Year Member



since there were no replies I'm trying a new thread, to better ask the questions I have

jdMorgan

2:38 am on Nov 18, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



To be specific, this will block requests from clients whose IP addresses start with 10. or 20. or 30. etc.

This code can be placed in the server-wide configuration file if you wish.

However, be aware that unless you make provisions for it, the code as written will prevent the use of custom 403 error documents, because there is no exception to allow those documents to be served. You may wish to consider using "Order Deny,Allow" and using a SetEnvIf directive to set a variable if a custom ErrorDocument is requested. Then use the "Allow from env=" construct to make an exception to allow the custom error document(s) to be served even to denied IP address ranges.

I suggest a similar exception for the robots.txt file. Some robots will just keep coming back trying to fetch robots.txt again and again if they receive anything but a 200-OK response to their requests for the robots.txt file.

If neither exception above is granted, then you may subject yourself to an essentially self-inflicted DOS attack when a robot tries to crawl your site from an unwelcome IP address range: Each request will trigger a cascade of 403 errors, and then that robot will keep coming back until it gets a 200-OK and a valid robots.txt file which asks it to go away...

Note that IP addresses are not assigned by country. Instead, in order to make our lives miserable and to make blocking troublemakers in an efficient manner as difficult as possible, they are assigned in (often very-small) "chunks" based only on the 'regional' assignment registries -- e.g. North America, South America, Europe, Asia/Pacific, Africa, etc., but with enough exceptions to make even those distinctions fairly meaningless. To block by country you will need to use GeoIP or Ip2Country lookups.

Jim

bbxrider

7:44 pm on Nov 20, 2010 (gmt 0)

10+ Year Member



thanks, this all sounds like a good approach and adds to the my project here, some of which I was working with another thread,
[webmasterworld.com ]