|Why is adsbot crawling?|
why is it crawling?
| 11:38 pm on Oct 8, 2013 (gmt 0)|
:: looking around uneasily because this is unfamiliar territory ::
Can someone explain in words of two syllables what this robot is looking for?
18.104.22.168 - - [07/Oct/2013:19:21:33 -0700] "GET /fonts/custom_greek_it.html HTTP/1.1" 200 6008 "-" "AdsBot-Google (+http://www.google.com/adsbot.html)"
The URL in the UA redirects to "Understanding landing page experience" which is, as far as I can make out, 100% irrelevant and unhelpful. I don't have an AdWords account. Not even in English, let alone Italian. Or, for that matter, AdSense. In any language. But this is about AdWords, isn't it?
| 5:13 pm on Oct 10, 2013 (gmt 0)|
In simple terms, that's usually for when checking the QS. Why its doing it is something more intriguing.
| 6:08 am on Oct 11, 2013 (gmt 0)|
You can stop AdsBot via robots file.
| 9:11 am on Oct 11, 2013 (gmt 0)|
|You can stop AdsBot via robots file. |
Sure, but that doesn't address the really tantalizing question of what the ### it's doing here in the first place :)
Nope, don't use GA either.
Do you suppose it's gathering ammunition for a targeted sales pitch? "Here is a site whose existence I am aware of, and it isn't attached to an AdThingummy account. This state of affairs cannot be allowed to continue."
| 12:02 am on Oct 31, 2013 (gmt 0)|
:: bump ::
Showed up again yesterday on the same page. I remain mystified :) Any chance it could be investigating links from some Italian page that does use AdThingie? Not that I've got any evidence such a page exists, unless it's an in-text link (meaning that visitors would show up as type-ins).
I've gone over and posted a question in google forums. But this frankly is an admission of defeat :(
| 4:30 am on Oct 31, 2013 (gmt 0)|
The AdsBot is generally checking for content on landing pages to determine relevance and quality score. I block it on GP because I am not running AdWords.
Many years ago I found an empty WP site bouncing an Ad campaign off my content and decided that it was easier to deny AdsBot. I have no clue how they were doing it, but a Google alert led me to the site and when I couldn't see anything there I used the URL to "save file as" and saw a .php header redirect. I am not saying that is what is happening, but if you are not running AdWords, I would deny it in robots.txt