jdMorgan
msg:407949 | 8:06 pm on Jun 1, 2003 (gmt 0) |
vik_c, No, robots.txt only works on robots that choose to read it and cooperate. You will need to block the IP address either in your configuration or by other means, such as "black-holing" that IP address at the firewall. The methods available to you depend on what server your site is hosted on, and you didn't say... Jim
|
DerekT
msg:407950 | 12:38 am on Jun 2, 2003 (gmt 0) |
You can ban any individul IP address or even a range or IPs using a .htaccess file on your server. Example
<Files .htaccess>
deny from all
</Files>
deny from 192.168.1.1
The first three lines prevent anyone from viewing your .htaccess file. The fourth line will deny the individual IP from your entire domain. You can also modify it to read: deny from 192.168.1. Which would deny all IPs in the range of 192.168.1.1 to 192.168.1.254
|
vik_c
msg:407951 | 1:15 pm on Jun 2, 2003 (gmt 0) |
Thanks Derekt, but that didn't work. JdMorgan, I use Apache.
|
DerekT
msg:407952 | 2:24 pm on Jun 2, 2003 (gmt 0) |
Im not sure how that wouldnt work. If you are running Apache it should work fine. Do you already have a .htaccess file in your root directory? Of course you will have to substitute the ip address in my example with the one causing you the problems.
|
jdMorgan
msg:407953 | 3:39 pm on Jun 2, 2003 (gmt 0) |
vik_c, Here are several methods to do it:
------
<Files .htaccess>
deny from all
</Files>
<Files *>
deny from 192.168.1.1
</Files>
------
SetEnvIf Remote_Addr ^192\.168\.0\.1$ banit
SetEnvIf Remote_Addr ^192\.168\.0\.8$ banit
SetEnvIf Remote_Addr ^192\.168\.0\.3$ banit
#
SetEnvIf Request_URI "^(/403.*\.html¦/robots\.txt)$" allowit
<Files *>
order deny,allow
deny from env=banit
allow from env=allowit
</Files>
------
RewriteCond %{REMOTE_ADDR} ^192\.168\.0\.1$ [OR]
RewriteCond %{REMOTE_ADDR} ^192\.168\.0\.8$ [OR]
RewriteCond %{REMOTE_ADDR} ^192\.168\.0\.3$
RewriteRule !^(robots\.txt¦403.*\.html)$ - [F]
------
Note that the latter two methods will allow access to robots.txt and to my custom 403 error pages, named "403.html" and "403info.html" --- If you use a custom 403 error page, you must allow even bad guys to access it. Otherwise, you set yourself up for an infinite loop when they visit, get a 403, redirect to the custom 403 error page, and get another 403 because they are blocked from accessing it... The IP addresses shown are simply placeholders - there are more efficient ways to ban IP address ranges. HTH,
Jim
|
vik_c
msg:407954 | 9:05 am on Jun 7, 2003 (gmt 0) |
Thanks Derek, JDMorgan, I wrote to my host and they banned the IP.
|
rbs10025
msg:407955 | 12:16 am on Jun 8, 2003 (gmt 0) |
<Files .htaccess>
deny from all
</Files>
|
|
Assuming your server is a relatively recent version of Apache (last year or two), this should not be necessary in your individual .htaccess as the Apache config file should already include it. Or more specifically, it says
<Files ~ "^\.ht">
Order allow,deny
Deny from all
Satisfy All
</Files>
|
|
which would also prevent people from snooping in .htpasswd files and etc.
|
|
