Welcome to WebmasterWorld Guest from 3.214.184.196

Forum Moderators: Ocean10000 & phranque

Message Too Old, No Replies

Unblocking Googlebot\-Image

one more rule in .HTACCESS - combo

     
3:48 pm on Aug 26, 2016 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2004
posts:1995
votes: 75


Currently I have a block of rules, all the way on top of .htaccess, that block Blank UA's and the ones that do not contain parentheses or contain an equal sign:


RewriteCond %{HTTP_USER_AGENT} ="" [OR]
RewriteCond %{HTTP_USER_AGENT} ^-$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^$ [OR]
RewriteCond %{HTTP_USER_AGENT} !\( [OR]
RewriteCond %{HTTP_USER_AGENT} !\) [OR]
RewriteCond %{HTTP_USER_AGENT} \=
RewriteRule !^blocked/ /blocked/ua/ [L,NC]

/blocked/ua/ [L,NC] directory records request and issues 403.

This is on IIS with ISAPI-Rewrite3

What I need to do is insert ONE more rule that does not block Googlebot-Image UA:
Something like a logical "AND NOT" after ALL of ORs are done or skip ALL ORs if the condition below evaluates to TRUE:
RewriteCond %{HTTP_USER_AGENT} !Googlebot\-Image

I tried a few things but not avail.

I also tried separating every [OR] and pairing with that last rule, it works, somewhat...

Anybody wants to take a stab at this one?

Thanks.
5:12 pm on Aug 26, 2016 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2004
posts:1995
votes: 75


never mind, I think... one extra NOT AND.

RewriteCond %{HTTP_USER_AGENT} ="" [OR]
RewriteCond %{HTTP_USER_AGENT} ^-$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^$ [OR]
RewriteCond %{HTTP_USER_AGENT} !\( [OR]
RewriteCond %{HTTP_USER_AGENT} !\) [OR]
RewriteCond %{HTTP_USER_AGENT} \=
RewriteCond %{HTTP_USER_AGENT} !RandomStringDoesNotMatterWhatItIs
RewriteCond %{HTTP_USER_AGENT} !Googlebot\-Image
RewriteRule !^blocked/ /blocked/ua/ [L,NC]

I think.
10:04 pm on Aug 26, 2016 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 893


one more rule in .HTACCESS
This is on IIS with ISAPI-Rewrite3

You're using htaccess on a IIS server?
2:20 am on Aug 27, 2016 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2004
posts:1995
votes: 75


Yes on 22 of them(Servers), along with web.config and urlrewrite module(IIS).

I am not an Apache person.
6:02 am on Aug 27, 2016 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 893


RewriteCond %{HTTP_USER_AGENT} !Googlebot\-Image
You shouldn't need to escape the dash (-) in Googlebot-Image. In fact all you need is the "Googlebot" part.

And just combining a couple lines, you could try this:
RewriteCond %{HTTP_USER_AGENT} ^-?$ [OR]
RewriteCond %{HTTP_USER_AGENT} ="" [OR]
RewriteCond %{HTTP_USER_AGENT} \=
RewriteCond %{HTTP_USER_AGENT} !\( [OR]
RewriteCond %{HTTP_USER_AGENT} !\) [OR]
RewriteCond %{HTTP_USER_AGENT} !(Googlebot|RandomString)
RewriteRule !^blocked/ /blocked/ua/ [L,NC]


However, your line...
RewriteCond %{HTTP_USER_AGENT} ^-$ [OR]
...should not block Googlebot-Image because it has an end anchor ($) meaning no more characters, so if your code was blocking Googlebot-Image, it is being blocked somewhere else. If your code was not blocking Googlebot-Image then leave out the !Googlebot-Image because it is not needed.