AdSense on pages without google visible incoming links?

[eRAZOR]

Let's say you've a site with lots of content but most of the content pages (about 8 million) are currently only accessible using the search page which GoogleBot obviously won't operate. Will AdSense still serve contextual ads on those pages?

[ThirdWheel]

The AS bots work independently of the SE bots, and will crawl any page that calls the AdSense JS. They will also crawl password protected areas providing you give AS account access details.

[eRAZOR]

That's what it thought. The problem is, the first time a page is hit, no ads will be displayed (at least from my experience), but later on when the page is revisited by a user there will be ads.

[ThirdWheel]

Have you opted for PSAs in the event that no ads are available? It takes a little while after the first crawl of a page for the contextual ads to be shown, so it needs to be established whether your alternate action is being executed when no ads are shown, or whether you think the AS code is failing. If it is just a case of alternate action for the first few impressions then I would say it's working okay.

If not, are your pages crawlable (eg, content not in iframes or generated by java), and is there a definite context? It's often tricky for the bot to determine this on sites such as blogs and forums. If the bot can determine a clear context, is this for a niche that attracts a lot of advertisers?

If you haven't done so already, try the AdSense preview tool and see if that displays relevant ads. Whatever the cause, it's unlikely it has anything to do with the fact that the pages are orphaned.

[eRAZOR]

Once a page has been visited at least once, ads are displayed correctly a short while after. Content is clean XTHML strict. The problem seems to soley in the fact that the pages do not have incoming links and are thus invisible to normal crawls. They only way to expose them to AdSense seems to be visiting them and by doing that calling the adsense javascript which in turns makes them known to the Adsense googlebot.

[edited by: eRAZOR at 10:50 am (utc) on Aug. 30, 2008]

[ThirdWheel]

That's how AS works, so I would say you don't have a problem. It is much more efficient for AS bots to only crawl pages that are visited, rather than seeking out which pages have AS on them and determing which ads to show in advance.

[himalayaswater]

You got 8 million pages in a one day?

[buckworks]

Have you opted for PSAs

With eight million pages, it could take a long time before any particular page gets a repeat visitor, so try not to waste the first impressions on PSA ads. Eight million impressions is a lot of exposure to give away.

Set up alternate ads of your own rather than opting for PSA's.

You could show affiliate banners, promote a charity of your own choosing, display something to promote one of your own products, or even just post navigation links to feature some other area of your site. Any of these would be better than giving away prime space to PSA's.

[fredw]

I'm not sure this is proven to help, but in my experience, it helps to have urls that contain keywords that are relative to my site. I do this, even though I don't really need that info to generate the page. For example, my page urls could be:

www.example.com/article.2008083001.html

but instead I make urls like:

www.example.com/news/widgets/blue-widgets/2008083001/widgets-now-more-widgety.html

and I never have a problem with untargeted ads on new pages. I always get properly targeted ads on the first view.

[ThirdWheel]

... try not to waste the first impressions on PSA ads

Wasn't advocating PSAs or otherwise, simply trying to establish to what exactly was meant by "no ads will be displayed". In my experience the three main issues are ...

1. AS account suspended
2. No relevant ads available and / or page not crawled
3. Incorrectly implemented AS code

... so thought we could try and nail it down.

I never have a problem with untargeted ads on new pages

Ads on new pages for sites that already carry AS often seem to be targeted immediately, perhaps because AS borrows the data from associated pages precrawl. If, however, you are able to introduce AS to a site that has never carried it and get targeted ads from impression #1 then I am deeply jealous / dubious (delete as applicable).

[eRAZOR]

buckworks: Good idea. I've setup the adsense units in question (actually I'm using Google Ad Manager) to alternatively display banners for some affiliate program.

[buckworks]

get targeted ads from impression #1

I've seen it happen. Keywords in the URL do seem to make a difference for the first impression. However, once the Adsense spider has visited the page, the page content will have the primary effect on what ads appear.

[ThirdWheel]

Keywords in the URL do seem to make a difference for the first impression

Wouldn't argue with that; I've certainly seen AS based solely on the URL until - as you say - the spider hits town. Hasn't worked favourably for me, but I guess, as ever, YMMV.

[swa66]

In my experience new pages get ads from the very first hit when the site is established and it'll use the topic of the site to give those ads. As it gets crawled the focus on the actual topic will come into play to make the ads more targeted.

But 8M pages ... that's got to smell (very) fishy over in thee Googleplex.