homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Google / Google AdWords
Forum Library, Charter, Moderators: buckworks & eWhisper & skibum

Google AdWords Forum

How Google Ad Bot Crawls a site

5+ Year Member

Msg#: 3389808 posted 6:37 pm on Jul 9, 2007 (gmt 0)

hi all,

we use paid search terms with google and have the google ad bot crawl our site now and again. WE are looking into creating dynamic content based on a users search. This is easy enough to do because we can parse the referrer information when a user clicks on a paid ad of ours and see what they searched for. Based on that we can dynamically create content relavant to their search.

However with the use of this type of system im forced to start thinking about page ranking with Google. The google Ad Bot crawls our sites however it does not leave a referrer footprint. Therefore i dont know what "paid keywords" its crawling the site for and therefore cannot display relavant content to the google bot.

I heard that if you do not have relavant content then it can hurt your page ranking and therefore cost more to purchase keywords. I was just wondering if there was a way for us to find out what the bot is looking for when it crawls our sites, so we can improve rankings and keep costs down.

Thanks in advance,



5+ Year Member

Msg#: 3389808 posted 2:10 pm on Jul 10, 2007 (gmt 0)

I'm not completely understanding your post. If you are running PPC ads then you want your landing pages to be relevant to your PPC ad copy and keywords. If you want to improve your quality score which will lower your CPC and help your ad position, create PPC ads that mirror the content you already have on your website. If you want to see what Google thinks of your landing pages, you can use their Keyword Tool and it will tell you the keywords it finds on your pages. Click on the Site-Related Keywords tab and enter your URL.


If you want to see what visitors searched for before they landed on your website, you can use a site stats package or Google Analytics.


5+ Year Member

Msg#: 3389808 posted 3:09 pm on Jul 10, 2007 (gmt 0)

thanks for the reply.

i realize how to make relavant static pages for our campaigns. However what we are planning is a totally dynamic site that creates content BASED on a users search terms. Therefore PAGES are not even created until a referrer object is parsed and business rules are applied to the terms serached, then content is pulled from XML and a database and written to the page to create "relavant" content.

However when the Ads Bot crawls our sites it does not pass referrer information so we just show it a "Default" page which in many cases does not contain the most releavant keywords, and information on it.

im just wondering if there is a way to know what the ads bot is looking for so our quality scores do not suffer.



5+ Year Member

Msg#: 3389808 posted 2:18 pm on Jul 11, 2007 (gmt 0)

If I knew the exact stuff it looks for, all of my as would score excellent on quality :-)


5+ Year Member

Msg#: 3389808 posted 2:19 pm on Jul 11, 2007 (gmt 0)

whoops, that should be ads, not as :-D


5+ Year Member

Msg#: 3389808 posted 4:33 pm on Jul 11, 2007 (gmt 0)

Hi Oxbaker,

We've been doing the exact same thing with AdWords for about 6 months now.

For many of our keywords, the best landing is a the search results we would show if the search had been performed on our internal engine. We create a destination URL that executes a search either exactly the same as, or very close to, the keyword.

We had similar concerns about the landing page component of our quality score. However, the Google bot seems to be able to access the search results page without a problem. The real question is, how good is the content on the search results page. Is there sufficient text for the Google bot to analyze?

Good luck!

Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google AdWords
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved