New posts sometimes indexed in minutes, sometimes not for a week

[wanderingmind]

I am sure many have seen some version of this.

New pages are added, Google pinged, and page appears in G index soon in minutes, and ranked too. That is normal for me.

However, once a month or once in two months, this does not happen -- new pages take a long time to show in up Google blogsearch after pinging, and will appear only in some DCs even after a week.

And just when one begins to get slightly mad, things revert back to normal. Max it takes ten days.

I often rely on quick stories written to capitalise on breaking news - and this depends on quick indexing and ranking which seems to work perfectly most of the time but inexplicably stops sometimes.

Others have told me the same often.

Any ideas on how I can avoid this?

[tedster]

Is this the same site you mentioned in the Mega Menu discussion [webmasterworld.com]? The one where you change home page links to 100 new stories on a daily basis? If so you may have an idea of where the problem comes from.

[wanderingmind]

Yup, it is :)

Though others who have complained about the same problem do not have sites like mine at all. SEOs I have talked to give me the basic stuff, and then tell me, its a Google thing, just wait it out. Always settles down. Yes it settles down alright. But not knowing for sure why is more a problem!

As its a breaking news site, changing that to a more static homepage with story links that stay for longer --- don't know how it would work, but if it really is the issue, will have to find a way.

For another site (not mine) it is a typical journal type blog, and things are the same for it - and they do not have the ever changing links I have.

Would there also be a site-specific limit to the number of new story pings one can send to Google, I wonder. Go over it, and you are frozen for some time.

[tedster]

Well, the number of pings is a different issue from the number of links on the home page. I've not heard of a limit on pings, but there well could be. Have you ever considered linking from the home page to category pages for the bulk of the new stories? I see major new outlets doing this, and as a user, I appreciate the approach. It also keeps total links per page to a more manageable level.

[wanderingmind]

True, that is a good option. In case of this problem re-occuring, that is what I will have to do I guess.

It though delays indexing - the fastest means of indexing (without links from external sites) seem to be publish, link from homepage, ping - if the link is not on the homepage, it doesn't seem to work.

Taking the link out of the homepage into categories might mean those pages would anyway get indexed later (thereby making the entire thing irrelevant!)

[tedster]

publish, link from homepage, ping

I thought publish/ping alone would do it. Staying in the index requires a link or two, but not the initial indexing. Maybe I'm wrong, but a number of people have reported that. Hard to sort out, since many CMSs add the new link to the home page automatically.