Blocking Google Snippet

[slipkid]

I am trying to block Google's snippet function in .htaccess and I'm having no success.

Here is a typical log entry:

66.249.84.5 - - [04/Apr/2013:01:56:56 -0400] "GET / HTTP/1.1" 200 5160 "-" "Mozilla/5.0 (Windows NT 6.1; rv:6.0) Gecko/20110814 Firefox/6.0 Google (+https://developers.google.com/+/web/snippet/)"

I have tried the following with no success:

SetEnvIf User-Agent ^\+https://www\.developers\.google\.com/\+/web/snippet/$ bad_bot

Can someone steer me in the right direction? Thanks.

[lucy24]

Why make it more complicated than it needs to be? If you're using mod_setenvif, a simple

BrowserMatch snippet bad_bot

should be enough.

Your pattern will always fail, because it has opening and closing anchors while the actual UA has more content both before and after.

[lucy24]

:: bump ::

Whoops! Now I need to go back and change the answer into a question. What is that
[developers.google.com...]
UA anyway? I've never set eyes on it until about a week ago, unless it was on a very long vacation. The web page leaves me none the wiser:

When a user shares a URL on Google+ or an app writes an app activity, Google+ attempts to fetch the content and create a snippet to provide a summary of the linked content.

Your web server will see a request with the user agent {et cetera}

Doesn't that make it sound like a preview? But all my logged snippets just pick up two things: the front page and the favicon. Thereby making mincemeat of my ordinary log-wrangling, which assumes that a page request followed by a favicon request is a human. The literal plus sign is a troublemaker too.

Hm. Maybe this all belongs in a different thread.

[Simon84]

Why on earth would you want to block this bot? This is Google coming along to build a proper preview for a link to your site that has been shared! If they get a better link preview to you, more people are likely to visit your site, thus more page views/conversions/users/whatever you're trying to acheive!

[lucy24]

The snippetbot is NOT a preview. Look at its behavior, not at its propaganda.

[slipkid]

There is little that Goofy does these days that does not set off warning bells.

Here are two examples of hits to raw logs:

66.249.84.5 - - [15/May/2013:02:11:23 -0400] "GET / HTTP/1.1" 200 5160 "-" "Mozilla/5.0 (Windows NT 6.1; rv:6.0) Gecko/20110814 Firefox/6.0 Google (+https://developers.google.com/+/web/snippet/)"

66.249.84.5 - - [15/May/2013:02:11:23 -0400] "GET /favicon.ico HTTP/1.1" 200 648 "-" "Mozilla/5.0 (Windows NT 6.1; rv:6.0) Gecko/20110814 Firefox/6.0 Google (+https://developers.google.com/+/web/snippet/)"

Without a referrer, I have no way of knowing which of my web pages is being rendered with a snippet per the above UA.

As far as I know, users of my site may be reading the snippet and going elsewhere.

At the moment, snippets are not blocked -- I am willing to change my code if this feature of Goofy is actually causing a loss of genuine visitors.