homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google AdWords
Forum Library, Charter, Moderators: buckworks & eWhisper & skibum

Google AdWords Forum

Why is adsbot crawling?
why is it crawling?

 11:38 pm on Oct 8, 2013 (gmt 0)

:: looking around uneasily because this is unfamiliar territory ::

Can someone explain in words of two syllables what this robot is looking for? - - [07/Oct/2013:19:21:33 -0700] "GET /fonts/custom_greek_it.html HTTP/1.1" 200 6008 "-" "AdsBot-Google (+http://www.google.com/adsbot.html)"

The URL in the UA redirects to "Understanding landing page experience" which is, as far as I can make out, 100% irrelevant and unhelpful. I don't have an AdWords account. Not even in English, let alone Italian. Or, for that matter, AdSense. In any language. But this is about AdWords, isn't it?



 5:13 pm on Oct 10, 2013 (gmt 0)

In simple terms, that's usually for when checking the QS. Why its doing it is something more intriguing.


 6:08 am on Oct 11, 2013 (gmt 0)

Using GA?

You can stop AdsBot via robots file.

User-agent: AdsBot-Google
Disallow: /


 9:11 am on Oct 11, 2013 (gmt 0)

You can stop AdsBot via robots file.

Sure, but that doesn't address the really tantalizing question of what the ### it's doing here in the first place :)

Nope, don't use GA either.

Do you suppose it's gathering ammunition for a targeted sales pitch? "Here is a site whose existence I am aware of, and it isn't attached to an AdThingummy account. This state of affairs cannot be allowed to continue."


 12:02 am on Oct 31, 2013 (gmt 0)

:: bump ::

Showed up again yesterday on the same page. I remain mystified :) Any chance it could be investigating links from some Italian page that does use AdThingie? Not that I've got any evidence such a page exists, unless it's an in-text link (meaning that visitors would show up as type-ins).

I've gone over and posted a question in google forums. But this frankly is an admission of defeat :(


 4:30 am on Oct 31, 2013 (gmt 0)

The AdsBot is generally checking for content on landing pages to determine relevance and quality score. I block it on GP because I am not running AdWords.

Many years ago I found an empty WP site bouncing an Ad campaign off my content and decided that it was easier to deny AdsBot. I have no clue how they were doing it, but a Google alert led me to the site and when I couldn't see anything there I used the URL to "save file as" and saw a .php header redirect. I am not saying that is what is happening, but if you are not running AdWords, I would deny it in robots.txt

Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google AdWords
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved