Lag between crawl and SERPS changes?

[graeme_p]

I have noticed before that site wide changes take a long time to be reflected in the SERPS.

Five days ago I made a design change that included adding a small navigation element to almost all pages on the site. Web Master Tools claims Googlebot is crawling hundreds of pages a day, and this is confirmed by the server logs, but if I do a search like:

site:example.com "my new navigation text"

I only get just over 200 results.

Is this usual? What is the point of Google crawling pages, if they then do not use the results?

I think it is possible that the design change (removal of ads plus new navigation) could affect ranking. How long should I give it before deciding that nothing has changed?

[g1smd]

Yes, it takes days to weeks for everything to filter in.

Be aware that title generation, snippet generation, page previews, cached page, content indexing, content ranking and link following are all separate processes that run independantly and the results are very often completely out of step with one another.

By looking very carefully at these things, you get some insight into how Google actually works. Pay attention to small details. Some are important.


Make sure you try both

site:www.example.com

and

site:example.com -inurl:www

searches.

Run Xenu LinkSleuth over your site and look carefully at the listings and the report.

Check how things are doing via Google WebmasterTools as well.

[RibaRiva]

I was dinged by the page-layout algo on January 19 although it took me a few weeks to figure out that's why my traffic was slipping. I removed the top banner on February 7 and only this weekend have I been seeing a traffic spurt despite my pages having been crawled several times this past month.

[Sgt_Kickaxe]

How long should I give it before deciding that nothing has changed?

As long as you can, I try to wait at least a month, but if there is no NEGATIVE effect you can wait 3-4 months or longer for any change to fully propagate.

A tip: when you make a major change to a page it can help to turn page cache off in google with a nocache meta tag(assuming pages were being cached before) and then when looking at serps you'll know all pages were updated when all of your cache versions are gone(no cache version link). You can then remove the meta tag and wait for them all to come back ensuring a full cycle has been completed before making more changes.

If you try that you need to make sure they are ALL gone and ALL come back, some pages take much longer than others.

[graeme_p]

I use nocache anyway. There is unlikely to be any negative effect on ranking, but the last lot of ads I took off has had a negative effect on income.

This is also partly in response to January 19th.

The site is canonicalised to a non-www version, so there is only one set of stuff to look at.

@g1smd, yes, thanks for pointing that out. I did not think about it, but it makes sense that its not just a division between crawling and generating SERPS, but multiple processes running their own schedules.

I have run a link checker recently. The only issues were some dead outgoing links (fixed last week).

I have also now requested faster crawl in WMT. Lets see if that makes a difference.