Forum Moderators: phranque

Message Too Old, No Replies

Unblocking Googlebot\-Image

one more rule in .HTACCESS - combo

         

blend27

3:48 pm on Aug 26, 2016 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Currently I have a block of rules, all the way on top of .htaccess, that block Blank UA's and the ones that do not contain parentheses or contain an equal sign:


RewriteCond %{HTTP_USER_AGENT} ="" [OR]
RewriteCond %{HTTP_USER_AGENT} ^-$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^$ [OR]
RewriteCond %{HTTP_USER_AGENT} !\( [OR]
RewriteCond %{HTTP_USER_AGENT} !\) [OR]
RewriteCond %{HTTP_USER_AGENT} \=
RewriteRule !^blocked/ /blocked/ua/ [L,NC]

/blocked/ua/ [L,NC] directory records request and issues 403.

This is on IIS with ISAPI-Rewrite3

What I need to do is insert ONE more rule that does not block Googlebot-Image UA:
Something like a logical "AND NOT" after ALL of ORs are done or skip ALL ORs if the condition below evaluates to TRUE:
RewriteCond %{HTTP_USER_AGENT} !Googlebot\-Image

I tried a few things but not avail.

I also tried separating every [OR] and pairing with that last rule, it works, somewhat...

Anybody wants to take a stab at this one?

Thanks.

blend27

5:12 pm on Aug 26, 2016 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



never mind, I think... one extra NOT AND.

RewriteCond %{HTTP_USER_AGENT} ="" [OR]
RewriteCond %{HTTP_USER_AGENT} ^-$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^$ [OR]
RewriteCond %{HTTP_USER_AGENT} !\( [OR]
RewriteCond %{HTTP_USER_AGENT} !\) [OR]
RewriteCond %{HTTP_USER_AGENT} \=
RewriteCond %{HTTP_USER_AGENT} !RandomStringDoesNotMatterWhatItIs
RewriteCond %{HTTP_USER_AGENT} !Googlebot\-Image
RewriteRule !^blocked/ /blocked/ua/ [L,NC]

I think.

keyplyr

10:04 pm on Aug 26, 2016 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



one more rule in .HTACCESS
This is on IIS with ISAPI-Rewrite3

You're using htaccess on a IIS server?

blend27

2:20 am on Aug 27, 2016 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Yes on 22 of them(Servers), along with web.config and urlrewrite module(IIS).

I am not an Apache person.

keyplyr

6:02 am on Aug 27, 2016 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



RewriteCond %{HTTP_USER_AGENT} !Googlebot\-Image
You shouldn't need to escape the dash (-) in Googlebot-Image. In fact all you need is the "Googlebot" part.

And just combining a couple lines, you could try this:
RewriteCond %{HTTP_USER_AGENT} ^-?$ [OR]
RewriteCond %{HTTP_USER_AGENT} ="" [OR]
RewriteCond %{HTTP_USER_AGENT} \=
RewriteCond %{HTTP_USER_AGENT} !\( [OR]
RewriteCond %{HTTP_USER_AGENT} !\) [OR]
RewriteCond %{HTTP_USER_AGENT} !(Googlebot|RandomString)
RewriteRule !^blocked/ /blocked/ua/ [L,NC]


However, your line...
RewriteCond %{HTTP_USER_AGENT} ^-$ [OR]
...should not block Googlebot-Image because it has an end anchor ($) meaning no more characters, so if your code was blocking Googlebot-Image, it is being blocked somewhere else. If your code was not blocking Googlebot-Image then leave out the !Googlebot-Image because it is not needed.