Welcome to WebmasterWorld Guest from 54.144.79.200

Forum Moderators: buckworks & eWhisper & skibum

Message Too Old, No Replies

Why is adsbot crawling?

why is it crawling?

     

lucy24

11:38 pm on Oct 8, 2013 (gmt 0)

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



:: looking around uneasily because this is unfamiliar territory ::

Can someone explain in words of two syllables what this robot is looking for?

72.14.199.209 - - [07/Oct/2013:19:21:33 -0700] "GET /fonts/custom_greek_it.html HTTP/1.1" 200 6008 "-" "AdsBot-Google (+http://www.google.com/adsbot.html)" 

The URL in the UA redirects to "Understanding landing page experience" which is, as far as I can make out, 100% irrelevant and unhelpful. I don't have an AdWords account. Not even in English, let alone Italian. Or, for that matter, AdSense. In any language. But this is about AdWords, isn't it?

engine

5:13 pm on Oct 10, 2013 (gmt 0)

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



In simple terms, that's usually for when checking the QS. Why its doing it is something more intriguing.

smallcompany

6:08 am on Oct 11, 2013 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



Using GA?

You can stop AdsBot via robots file.

User-agent: AdsBot-Google
Disallow: /

lucy24

9:11 am on Oct 11, 2013 (gmt 0)

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



You can stop AdsBot via robots file.

Sure, but that doesn't address the really tantalizing question of what the ### it's doing here in the first place :)

Nope, don't use GA either.

Do you suppose it's gathering ammunition for a targeted sales pitch? "Here is a site whose existence I am aware of, and it isn't attached to an AdThingummy account. This state of affairs cannot be allowed to continue."

lucy24

12:02 am on Oct 31, 2013 (gmt 0)

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



:: bump ::

Showed up again yesterday on the same page. I remain mystified :) Any chance it could be investigating links from some Italian page that does use AdThingie? Not that I've got any evidence such a page exists, unless it's an in-text link (meaning that visitors would show up as type-ins).

I've gone over and posted a question in google forums. But this frankly is an admission of defeat :(

not2easy

4:30 am on Oct 31, 2013 (gmt 0)

WebmasterWorld Administrator 5+ Year Member Top Contributors Of The Month



The AdsBot is generally checking for content on landing pages to determine relevance and quality score. I block it on GP because I am not running AdWords.

Many years ago I found an empty WP site bouncing an Ad campaign off my content and decided that it was easier to deny AdsBot. I have no clue how they were doing it, but a Google alert led me to the site and when I couldn't see anything there I used the URL to "save file as" and saw a .php header redirect. I am not saying that is what is happening, but if you are not running AdWords, I would deny it in robots.txt
 

Featured Threads

Hot Threads This Week

Hot Threads This Month