my htaccess format needs coding help to block Agent: Webgo

[Joey123]

I want to block this agent:

" WebGo IS - 2536 "

It appears above exactly as it is in my logs. Here is how my .htaccess file looks like in part.

RewriteEngine on
RewriteBase /
RewriteCond %{REQUEST_URI} \.htaccess$ [OR]
RewriteCond %{HTTP_USER_agent} ^Feedster_Crawler [NC,OR]
RewriteCond %{HTTP_USER_agent} ^Twiceler [NC,OR]
RewriteCond %{HTTP_USER_agent} ^MFC_Tear_Sample [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailCollector
RewriteRule ^.* - [F]

How do I add the agent "webgo" in a similar format without having to alter the way I have been banning agents in my htaccess file? The spaces in between the agent's words, even if I use the underscore, does not get it blocked.

Can someone edit this for me? Again, I do not want to change my existing format.

Thank you very much for your "free" time.

[jd01]

Hi Joey,

RewriteEngine on
RewriteBase /
RewriteCond %{REQUEST_URI} \.htaccess$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^Feedster_Crawler [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Twiceler [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^MFC_Tear_Sample [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailCollector
RewriteRule ^.* - [F]

The spaces are the problem. There are a couple of things you can do:
1. Use the implicit 'and everything else'

RewriteCond %{HTTP_USER_AGENT} ^WebGo [NC,OR]

2. Escape the spaces in the UA string

RewriteCond %{HTTP_USER_AGENT} ^WebGo\ IS\ -\ 2536 [NC,OR]

These should both work if the UA string starts with WebGo. If not you will need to add the begining string or remove the preceding ^ from the rule to use the implicit 'anything before...'

The final ruleset will look something like this:

RewriteEngine on
RewriteBase /
RewriteCond %{REQUEST_URI} \.htaccess$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebGo [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Feedster_Crawler [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Twiceler [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^MFC_Tear_Sample [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailCollector
RewriteRule . - [F]

I also made a couple of corrections:
1. Adjusted the word agent to proper case.
2. Removed the hard beginning ^ and the catch all .* for efficiency. .(dot) checks for any single character in a request string, and is adequate for the purpose of this ruleset.

Hope this helps.

Justin

[jd01]

Little error correction -

The rule should be:

RewriteRule .* - [F]

Forgot / does not show as a character in .htaccess comparrison, so technically we need the optional 0 of .*

Justin

[Joey123]

Thank you for the assist.

RewriteCond %{HTTP_USER_AGENT} ^Feedster_Crawler [NC,OR]

1) What about the underscore above? I put it there to fix the space. Will that work like \ or do I need to go back and replace all my underscores?

2) Could you tell me about one more thing? Below is what my full htaccess file looks like. I put asterisks to denote the IP's. Are there any problems?

SetEnvIf Request_URI "(robots\.txt)$" allowit

<Files *>
Order Deny,Allow
#sitename1
Deny from 64.***.**.0/18
Deny from 216.**.***.0/19
#sitename2
Deny from **.148.**.0/27
Deny from ***.**3.0.0/16
Deny from 65.***.**.0/27
Allow from env=allowit
</Files>

RewriteEngine on
RewriteBase /
RewriteCond %{REQUEST_URI} \.htaccess$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla.*NEWT [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebGo\ IS\ -\ 2536 [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailCollector
RewriteRule .* - [F]

[jdMorgan]

If you want to match a space, then use the "\ " pattern; Underscores and spaces are not equivalent.

If you've seen both underscores and spaces in actual logged user-agent names, then you could either use "[\ _]" or "(\ ¦_)" as a pattern. Or if there are many variations, you could just use "." to match any single character.

Be aware the pipe characters "¦" (meaning logical "OR") are modified when posting on this forum. You will need to edit them in any code copied and pasted from here, and change them to solid pipe characters from your keyboard -- usually Shift-\ on a U.S. 101-key keyboard.

I don't see any problems with your code.

Jim