| 3:59 pm on Mar 26, 2008 (gmt 0)|
This happens a lot when a site has many pages that might answer a given search. The previously unranked url was probably the third best from your domain for that particular search.
There is a very basic Google filter that only allows two urls from any given domain in the entire result set. Even if another url is third best, and very close to the indented url in ranking and relevance, it cannot show up on a keyword search.
So if two different urls from the same domain would be ranked on the same page, they will be clustered with the second one indented. They won't rank at #3 and #10, for example (unless you tweak the search results down to 5 per page.)
If the ranking shifts for any reason - an algo tweak or change in backlinks or even a page-specific penalty, then the previously third best can move up to being the second best, and the previously indented result now gets filtered out.
| 4:36 pm on Mar 26, 2008 (gmt 0)|
I was surprised that a page that had previously not ranked in the top 30 for the query was able to immediately replace the page that is no longer ranking.
I always assumed that when the page was no longer relevant to the query, it would drop out and the indented listing would simply disappear, not be replaced by the next most relevant page from the domain for the query which seems to be the case here. Now, the new page that as mentioned above wasn't top 30 before is legitimately ranking #4 as I looked at the SERPS with only 4 results per page and the home page was #3 and this page was indented at #4.
I did a 'site:domain.com keyword' query in Google and the page that was previously ranking in the indented spot is listed at #2, making me think it's the second most relevant page from the domain regarding the query, but in the real SERPS it is nowhere to be found and the page that is ranked 3rd in the 'site:domain.com keyword' query is in the indented listing in the actual SERPS for the keyword. So it seems like Google is seeing the page in the domain search, but is not ranking it. It's also not appearing in the cache for some reason and I checked and I didn't see any 'noarchive' tag or anything that would cause that.
Either I'm blatantly missing something here that would cause this (very likely) or something fishy is going on with the index and the SERPS (possible), and if you can spot it please point it out.
| 4:44 pm on Mar 26, 2008 (gmt 0)|
What do you see when you do the keyword search, and then append &filter=0 to the end of the resulting url? Especially if you set results to 100 per page?
| 6:02 pm on Mar 26, 2008 (gmt 0)|
When it is 10 results per page with &filter=0, I have the 3rd (homepage),4th (new page), and 5th (old page that I was aggressively building links for) spots, and when it is 100 per page I have 6 or 7 listings.
What's odd though, is when I do the normal query without &filter=0 and then click on the 'More results from www.domain.com' the page that was there previously (aggressive linked page) is 2nd, but replaced in the SERPS by the new page.
Not sure if this is a filter or something else entirely different.
| 6:11 pm on Mar 26, 2008 (gmt 0)|
Well, Google does monitor the speed of link growth. It was one of a big pile of factors mentioned in the 2005 History and Age Data patent [appft1.uspto.gov].
| Consider the example of a document with an inception date of yesterday that is referenced by 10 back links. This document may be scored higher by search engine than a document with an inception date of 10 years ago that is referenced by 100 back links because the rate of link growth for the former is relatively higher than the latter. |
While a spiky rate of growth in the number of back links may be a factor used by search engine to score documents, it may also signal an attempt to spam search engine. Accordingly, in this situation, search engine may actually lower the score of a document(s) to reduce the effect of spamming.
You may have smacked into the mechanism that handles link growth that looks too fast - beyond a certain multiple of the standard deviation, for instance.
| 6:18 pm on Mar 26, 2008 (gmt 0)|
Yeah, this is what I originally thought had happened, but I wanted to see if there were any other reasons why this might have occured.
Since this is so new (just noticed it last night) I am not going to take any action for at least a week or two to see if these changes are permanent. If this is still in effect in a few weeks, do you think it is something that will self-correct as the link acquisition rate slows with time, or should I actively go about removing or re-targeting links that we acquired?
| 6:20 pm on Mar 26, 2008 (gmt 0)|
I'd lean toward waiting for self-correction - especially since we're both just taking educated guesses. Even if we've got it right, it sounds like you're just over the edge of the envelope and a little more time with no further link growth could bring it back.
| 7:07 pm on Mar 27, 2008 (gmt 0)|
The page in question has been re-added. Everything is back to normal regarding that page. Oddly though there was no cache of the page yesterday but there is one today and it's dated 3/20.
And I've found 3 other pages from the site that are no longer in the index today, and also have no cached copies. Still not sure what's going on here.
Has anybody else seen pages and cached versions be temporarily removed from the index only to be added back within 48 hours?