Forum Moderators: buckworks & skibum

Message Too Old, No Replies

Robots.txt vs. Meta Tag Bot Blocking for Adsbot-Google

Robots.txt Meta Bot Blocking Adsbot-Google

         

JimmyH

12:10 am on Nov 20, 2007 (gmt 0)

10+ Year Member



I've been using meta tags to block bots from indexing my landing pages so I don't get penalized for duplicate content.

I use the tag NAME="ROBOTS" CONTENT="NOINDEX,NOFOLLOW"

1 - Does anyone know if this will also block Adsbot-Google from assessing the quality score of my pages?
2 - Is there an advantage to using robots.txt over meta tags?

Thanks.

koncept

3:00 am on Nov 20, 2007 (gmt 0)

10+ Year Member



What kind of quality score do you think Google will give you when the adbot discovers you have blocked it?

JimmyH

3:48 am on Nov 20, 2007 (gmt 0)

10+ Year Member



Sorry - to clarify, I do NOT want to block Adsbot-Google. I want to make sure that this bot sees my pages.

koncept

5:08 am on Nov 20, 2007 (gmt 0)

10+ Year Member



Oops, sorry Jimmy!

smallcompany

5:41 am on Nov 20, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Not sure about meta tags, but here is what you can do with robots.txt, in regards of AdWords specifically (from AdWords' online help):

-----
While we strongly recommend against restricting our system's automatic review of your landing page, you can edit your site's robots.txt file to avoid a review. The file must explicitly exclude your page from our system visits as follows:

To prevent AdsBot-Google from accessing your site, add the following to your robots.txt file:

User-agent: AdsBot-Google
Disallow: /

To prevent AdsBot-Google from accessing parts of your site, add the following to your robots.txt file:

User-agent: AdsBot-Google
Disallow: /exclude/

Where exclude represents the directories you don't want the AdWords system to visit.

Note: In order to avoid increasing CPCs for advertisers who don't intend to restrict AdWords visits to their pages, the system will ignore blanket exclusions (User-agent: *) in robots.txt files.
--------

Note tells you that they will ignore the *, which is used if you are blocking all search bots, which should work for you.