Forum Moderators: phranque
When using an HTTP viewer it returns a 403 for all combinations of the above IP, ie for 000.000.000.01
BUT when I look at my logs these IP's get a 200 header and still crawl my site. What can I do now?
RewriteCond %{HTTP_USER_AGENT} ^EmailCollector [OR]
RewriteCond %{HTTP_USER_AGENT} ^wells [OR]
RewriteCond %{HTTP_USER_AGENT} ^Missigua [OR]
RewriteRule ^.* - [F]
[added]
Also, you must escape the literal periods in your patterns to avoid them being treated as "any single character" regex tokens, and start-anchor the patterns. Not doing this would lead to possibly-dangerous ambiguities, should there be any single-digit octets in the actual address(es).
RewriteCond %{REMOTE_ADDR} ^1\.2\.3\.
Jim
[edited by: jdMorgan at 2:20 pm (utc) on Jan. 29, 2008]
For the future: not sure I understand correctly how to ban an IP range. would it work like this?
to block: aaa.bbb.ccc.0 to aaa.bbb.ccc.999
RewriteCond %{REMOTE_ADDR} ^aaa\.bbb\.ccc\.
RewriteRule - [F]
Thanks again
For the future: not sure I understand correctly how to ban an IP range. would it work like this?to block: aaa.bbb.ccc.0 to aaa.bbb.ccc.999
RewriteCond %{REMOTE_ADDR} ^aaa\.bbb\.ccc\.
RewriteRule - [F]
Thought I had an old Webmaster World bookmark which provided explantion of IP ranges and their use in the various Class groups.
Others may explain this better, however here goes.
The Class groups are not numerics, rather charcters.
The range of characters are 0-255
Using the example that you've provided would deny access (note exception below) to the 0-255 range (in Class D) or what ever Class C that you've specifed as "ccc".
Potentially, each Class (whether A, B, C or D) offers the following ranges:
[0-9]¦[1-9][0-9]¦1[0-9][0-9]¦2[0-5][0-9]
(Please note the forum breaks these pipe characters and they requiring editing before use).
broken down as follows:
0-9, 10-99, 100-199, 200-259
With the 250-259 expression being used to save expression space, even though it defines non-existent ranges (256-259).
It would be redundant to specify the entire 0-255 range a Class D expression, however it would still work.
You may break these numbers down into smaller groups for broader expressions.
EX:
aaa.bbb.ccc.[0-9] which would deny the 0-9 Class D range of the Class C that "ccc" represents.
Another example is aaa.(bbb¦bab)\.ccc.[0-9]
which would provide an OR for two different Class B ranges represented by "bbb or bab" with the Class C range being the same in both instance, while the Class D range of 0-9 is also the same in both instances.
(exception)
The folllowing is what I use, the logic has long since passed from my recollection.
RewriteRule .* - [F]
RewriteCond %{REQUEST_URI}!^/(errors/403\.htm¦errors/404\.htm¦errors/500\.htm¦robots\.txt)$
RewriteCond %{REMOTE_ADDR} ^aaa\.bbb\.ccc\. [OR]
RewriteCond %{HTTP_USER_AGENT} SomeBot
RewriteRule ^.* - [F]
Did I overlook something? Or are there other reasons why this might not work?
You can see in the following thread what it is I am trying to block, in case it is relevant: [webmasterworld.com...]
(As this might be a search engine spider, I probably won't leave the block in my htaccess, but would really like to know what I am doing wrong here).
Still the named bot is blocked, but the ip range is not. I tried it with only the block for the ip range and also only blocking a single ip, but it always returns a 200.
Here is what I currently have in htaccess regarding this, after having made the changes according to your replies:
Are the 200's a result of redirects to either 403 or 404 pages?
Check your visitor logs and compare files sizes for the actual requested pages to the 200's of these denied ranges.
In addition, you do not provide if these NEW Rewrites are the solitray lines in your htaccess?
As a result, I ponder if you've opened the lines with:
RewriteEngine on
RewriteCond %{REQUEST_URI}!^/(errors/403\.htm¦errors/404\.htm¦errors/500\.htm¦robots\.txt)$
RewriteCond %{REMOTE_ADDR} ^aaa\.bbb\.ccc\. [OR]
RewriteCond %{HTTP_USER_AGENT} SomeBot
RewriteRule ^.* - [F]
joergnw10,
The lines contained in my htaccess are rather simple compared to many of the inquiries and answers that I see here.
As a result, there are corners of Apache and regex that simply fail to gather my attention or even understanding.
Another will need to advise of the effect of the following line:
RewriteCond %{REQUEST_URI}!^/(errors/403\.htm¦errors/404\.htm¦errors/500\.htm¦robots\.txt)$
In addition, I'm not all that sure that it's possible to combine UA's and IP ranges and achieve denail in the manner you have?
Try the following and see if two rewrites solve your issue:
#Rewrite 1
RewriteCond %{REQUEST_URI}!^/(errors/403\.htm¦errors/404\.htm¦errors/500\.htm¦robots\.txt)$
RewriteCond %{REMOTE_ADDR} ^aaa\.bbb\.ccc\.
RewriteRule ^.* - [F]
#Rewrite 2
RewriteCond %{REQUEST_URI}!^/(errors/403\.htm¦errors/404\.htm¦errors/500\.htm¦robots\.txt)$
RewriteCond %{HTTP_USER_AGENT} SomeBot
RewriteRule ^.* - [F]
[edited by: wilderness at 2:40 pm (utc) on Feb. 3, 2008]
You can see in the following thread what it is I am trying to block, in case it is relevant: [webmasterworld.com...]
(As this might be a search engine spider, I probably won't leave the block in my htaccess, but would really like to know what I am doing wrong here).
The person who initiated the example in this thread, never actually provided the 208 range, anther did!
Thus how would anybody realize the accuracy?
However to deny the range that was provided in the answer?
RewriteCond %{REMOTE_ADDR} ^207\.179\.(12[89]¦1[3-8][0-9]¦19[01])\.
RewriteRule .* - [F]
Correct broken pipe characters.
As an aside, that a webmaster would place emphasis on the primary contents of google analytics reports, which are created to cover a group of the masses (much the same as the stats reports that are used by host providers), without determining the location of and learning how to interpret the content of their website visitors logs?
Is an asburd idea, from, my point of view.
My typo.
Sorry.
Two typos.
This should actually read:
RewriteCond %{REMOTE_ADDR} ^208\.111\.(12[89]¦1[3-8][0-9]¦19[01])\.
RewriteRule .* - [F]
RewriteCond %{REQUEST_URI}!^/(errors/403\.htm¦errors/404\.htm¦errors/500\.htm¦robots\.txt)$
Jim
