Welcome to WebmasterWorld Guest from

Forum Moderators: buckworks & eWhisper & skibum

Message Too Old, No Replies

How Google Ad Bot Crawls a site



6:37 pm on Jul 9, 2007 (gmt 0)

5+ Year Member

hi all,

we use paid search terms with google and have the google ad bot crawl our site now and again. WE are looking into creating dynamic content based on a users search. This is easy enough to do because we can parse the referrer information when a user clicks on a paid ad of ours and see what they searched for. Based on that we can dynamically create content relavant to their search.

However with the use of this type of system im forced to start thinking about page ranking with Google. The google Ad Bot crawls our sites however it does not leave a referrer footprint. Therefore i dont know what "paid keywords" its crawling the site for and therefore cannot display relavant content to the google bot.

I heard that if you do not have relavant content then it can hurt your page ranking and therefore cost more to purchase keywords. I was just wondering if there was a way for us to find out what the bot is looking for when it crawls our sites, so we can improve rankings and keep costs down.

Thanks in advance,


2:10 pm on Jul 10, 2007 (gmt 0)

5+ Year Member

I'm not completely understanding your post. If you are running PPC ads then you want your landing pages to be relevant to your PPC ad copy and keywords. If you want to improve your quality score which will lower your CPC and help your ad position, create PPC ads that mirror the content you already have on your website. If you want to see what Google thinks of your landing pages, you can use their Keyword Tool and it will tell you the keywords it finds on your pages. Click on the Site-Related Keywords tab and enter your URL.


If you want to see what visitors searched for before they landed on your website, you can use a site stats package or Google Analytics.


3:09 pm on Jul 10, 2007 (gmt 0)

5+ Year Member

thanks for the reply.

i realize how to make relavant static pages for our campaigns. However what we are planning is a totally dynamic site that creates content BASED on a users search terms. Therefore PAGES are not even created until a referrer object is parsed and business rules are applied to the terms serached, then content is pulled from XML and a database and written to the page to create "relavant" content.

However when the Ads Bot crawls our sites it does not pass referrer information so we just show it a "Default" page which in many cases does not contain the most releavant keywords, and information on it.

im just wondering if there is a way to know what the ads bot is looking for so our quality scores do not suffer.



2:18 pm on Jul 11, 2007 (gmt 0)

5+ Year Member

If I knew the exact stuff it looks for, all of my as would score excellent on quality :-)


2:19 pm on Jul 11, 2007 (gmt 0)

5+ Year Member

whoops, that should be ads, not as :-D


4:33 pm on Jul 11, 2007 (gmt 0)

5+ Year Member

Hi Oxbaker,

We've been doing the exact same thing with AdWords for about 6 months now.

For many of our keywords, the best landing is a the search results we would show if the search had been performed on our internal engine. We create a destination URL that executes a search either exactly the same as, or very close to, the keyword.

We had similar concerns about the landing page component of our quality score. However, the Google bot seems to be able to access the search results page without a problem. The real question is, how good is the content on the search results page. Is there sufficient text for the Google bot to analyze?

Good luck!


Featured Threads

Hot Threads This Week

Hot Threads This Month