New Pages - Now you see them - Now you don't

[elaineb]

Just over this last week I've created 3 new pages, which Google indexed the next day and all appeared in the top 5 for their main keyword - great.

Then a couple of days later they all disappear and they don't even appear when I check the 'cached date' -
'your search #*$!xx did not match any documents'

The site in question is a long established site with thousands of main keywords bobbing about on the first page of Google - and these 3 pages should have been straight in there.

The only thing different is I forgot to add them to the Google sitemap and my own sitemap - would this have made them disappear?

[tedster]

This is one of several similar reports I've seen. It sounds like a sandbox effect on new domains, but occuring just for newly published directories or even a few new pages on long-established domains.

Don't know what to say about it. I haven't seen anything like this for sites I work with, and we certainly do publish new urls with new content regularly.

My suspicion is that Google has your new urls only at some data centers, not others, and as you may know, every time you access google.com you are playing a kind of data center roulette. If that's the case, the whole thing will probably fix itself in a few days.

Another scenario is also possible -- these urls are too remotely linked to, considering the amount of PR that the rest of the urls in the domain are "circulating".

[edited by: tedster at 12:31 am (utc) on Oct. 11, 2006]

[Pirates]

Were the new pages in your template and linked too by every page in the site?

[mcskoufis]

Something similar happened to me... I uploaded on a subdomain extracts from some scientific research I've recently finished, each with about 1500 words in them. Google indexed the main page about a week ago (there is a PR6 wiki link and a sitewide link from my main personal site.

Things at Google seem to be in constantly updating... All site: link: inurl: and the toolbar's pagerank seem to be very unstabilised at the moment. Bit like the weather in Britain...

I reckon we should definitely not deploy any desperate measures until things stabilise.

[g1smd]

Google has several versions of their index in play, and I notice that one version doesn't seem to update as quickly as the others. Can you try several datacentres and see if you show up on some but not others? You don't need to try all 44, just a few will likely show up some differences.

[Pirates]

there is a PR6 wiki link and a sitewide link from my main personal site

That would make sense a new page with so many links I think would get sandboxed untill there was an equilibrium of the keyword throughout the site. ie an expansion of the category.

[elaineb]

The new pages are highly relevant to the theme of the site - I never try to con Google, way too risky.

Thought about trying for different data centres, but it's a UK site and doesn't always show for top positions in the generic ones.

Don't use a template as such, but the page structure is obviously the same. The pages have gone into my ROS nav bar - always do, and this has always proved beneficial.

The new pages are all hand produced, no dynamic or data feeds, and I tend to create new pages often and this has never happened before.

It's been a week now and still no sign of them in the cache - strange.

[mcskoufis]

That would make sense a new page with so many links I think would get sandboxed untill there was an equilibrium of the keyword throughout the site. ie an expansion of the category.

I forgot to mention that the site is a subdomain of my personal site... It is not a site-wide from a totally unrelated site... It is part of it..

[Halfdeck]

It could be you're hitting different DCs. It could also be your new pages are what Matt Cutts would refer to as being on the "fringe," saddling the fense. Try pointing a few urls at them to tip them over into the main index?

[steveb]

I see this too. Pages are added as "fresh" but then fall out of the index the next day.

Google's new crawl is just way inferior to what it used to be. They'll get in eventually but the weaklingbot just isn't as good as the old one.

[elaineb]

well the new pages returned Thursday and with some good keyword rangings which resulted in extra visitors - great.

Today, Saturday - they've disappeared again.

Had a look at my site:info - and for the first time in ever there's no supplementals listed - or extra pages - GG had a habit of indexing 2 pages together! - very, very clean site info now.

Googlebot must be too busy cleaning up to index new pages properly - just got to be patient.