Welcome to WebmasterWorld Guest from 18.104.22.168
I was building links rather aggressively for this term and believe I may have aligned the page too closely with the keyword phrase and may have had it filtered out.
Anybody else have thoughts about this, seen this happen, or have other conjectures for why this might occur?
There is a very basic Google filter that only allows two urls from any given domain in the entire result set. Even if another url is third best, and very close to the indented url in ranking and relevance, it cannot show up on a keyword search.
So if two different urls from the same domain would be ranked on the same page, they will be clustered with the second one indented. They won't rank at #3 and #10, for example (unless you tweak the search results down to 5 per page.)
If the ranking shifts for any reason - an algo tweak or change in backlinks or even a page-specific penalty, then the previously third best can move up to being the second best, and the previously indented result now gets filtered out.
I was surprised that a page that had previously not ranked in the top 30 for the query was able to immediately replace the page that is no longer ranking.
I always assumed that when the page was no longer relevant to the query, it would drop out and the indented listing would simply disappear, not be replaced by the next most relevant page from the domain for the query which seems to be the case here. Now, the new page that as mentioned above wasn't top 30 before is legitimately ranking #4 as I looked at the SERPS with only 4 results per page and the home page was #3 and this page was indented at #4.
I did a 'site:domain.com keyword' query in Google and the page that was previously ranking in the indented spot is listed at #2, making me think it's the second most relevant page from the domain regarding the query, but in the real SERPS it is nowhere to be found and the page that is ranked 3rd in the 'site:domain.com keyword' query is in the indented listing in the actual SERPS for the keyword. So it seems like Google is seeing the page in the domain search, but is not ranking it. It's also not appearing in the cache for some reason and I checked and I didn't see any 'noarchive' tag or anything that would cause that.
Either I'm blatantly missing something here that would cause this (very likely) or something fishy is going on with the index and the SERPS (possible), and if you can spot it please point it out.
What's odd though, is when I do the normal query without &filter=0 and then click on the 'More results from www.domain.com' the page that was there previously (aggressive linked page) is 2nd, but replaced in the SERPS by the new page.
Not sure if this is a filter or something else entirely different.
 Consider the example of a document with an inception date of yesterday that is referenced by 10 back links. This document may be scored higher by search engine than a document with an inception date of 10 years ago that is referenced by 100 back links because the rate of link growth for the former is relatively higher than the latter.
While a spiky rate of growth in the number of back links may be a factor used by search engine to score documents, it may also signal an attempt to spam search engine. Accordingly, in this situation, search engine may actually lower the score of a document(s) to reduce the effect of spamming.
You may have smacked into the mechanism that handles link growth that looks too fast - beyond a certain multiple of the standard deviation, for instance.
Since this is so new (just noticed it last night) I am not going to take any action for at least a week or two to see if these changes are permanent. If this is still in effect in a few weeks, do you think it is something that will self-correct as the link acquisition rate slows with time, or should I actively go about removing or re-targeting links that we acquired?
And I've found 3 other pages from the site that are no longer in the index today, and also have no cached copies. Still not sure what's going on here.
Has anybody else seen pages and cached versions be temporarily removed from the index only to be added back within 48 hours?