Forum Moderators: Robert Charlton & goodroi
In addition to history of positions (or rankings) of documents for a given query, search engine 125 may monitor (on a page, host, document, and/or domain basis) one or more other factors, such as the number of queries for which, and the rate at which (increasing/decreasing), a document is selected as a search result over time; seasonality, burstiness, and other patterns over time that a document is selected as a search result; and/or changes in scores over time for a URL-query pair.USPTO page [appft1.uspto.gov]
I noticed the patent is not even talking about clicks or actual traffic here. It's talking about how often the document is even chosen for placement in the the search results -- impressions. So might Google be moving a page back into the Supplemental Index if it just doesn't come up "often enough" for any searches - by whatever threshhold number Google sets?
[edited by: tedster at 6:31 pm (utc) on June 26, 2007]
Also if I was a Googler, I wouldn't put too much weight on CTR anyway. It just means the title and description snippets are on target. A highly relevant, useful page with a lousy TITLE tag might never get clicked on.
What's more important to me is whether or not a user returns from a search result. That's a big sign that the user didn't find what he/she was looking for.
I would think a page not being clicked on would just drop in ranking
I agree with you - but this patent is not just talking about CTR. As I read it, it's also talking about recording whether any query even taps that URL for DISPLAY, with no regard for whether there's a click.
My thinking is that if end-users rarely formulate a search that chooses a certain URL, then even if other quality factors are in place, Google may not want to include such a URL in their main index for performance reasons.
Matt Cutts recently explained that Supplemental URLs are indexed differently, with only certain keywords being flagged rather than a full text indexing. So I'm thinking that Google might move such a URL to the Supplemental Index and just flag those few search terms that have brought it up once in a while. That would save on resources -- storage, indexing and frequent crawling -- and still allow the URL to show in the few SERPs where it's needed.
Something like this could explain some of the stranger migrations I've seen, back and forth from Supplemental to Main to Supplemental.
[edited by: tedster at 6:38 am (utc) on June 25, 2007]
My thinking is that if end-users rarely formulate a search that chooses a certain URL, then even if other quality factors are in place, Google may not want to include such a URL in their main index for performance reasons.
Good point. I do have pages targeting nonsensical terms in the main index that hardly gets any clicks/views, but the thought have crossed my mind.
I also notice Google going out of its way to rank pages that don't contain all the words in a query instead of ranking a supplemental page targeting a long tail.
For example, "blue widgets red boots sunday california idaho pointy hats" bring up 100 pages of results from the main page. None of them are supplemental. This inspite the fact that Googlers hinted at more traffic from supplemental results in the future.
Relevance, in cases like this, seems to be less important to ranking than page "importance."
Even if Google filtered out pages with infrequent views/CTR, their results probably won't look much different than it does now.
[edited by: Halfdeck at 5:58 am (utc) on June 25, 2007]
So might Google be moving a page back into the Supplemental Index if it just doesn't come up "often enough" for any searches - by whatever threshhold number Google sets?
An intriguing thought. My hunch is, though, that rather than one factor, there's a "Supplemental algo" that determines what goes into that index.
Google has said that the core factor for Supplementals is low PageRank. I know of many high PR pages that target some very rarely searched phrases (eg, cutting edge technology), and I don't see them going Supplemental.
"Supplemental" is probably too strong a word for the kind of seasonality and burstiness factors that the patent most probably is used for, but these could be part of the monitoring continuum that includes Supplementals at the low end.
Google likes solutions that scale, and, as "all of the world's information" continues to grow, they're going to need to use every scaling trick they can patent.
[edited by: tedster at 6:55 am (utc) on June 26, 2007]
So those pages pages would come up in the search - possibly even at #1, and even if they were assigned supplemental status.
This is what I have seen lately. It doesn't make any difference whether the page has a normal or supplemental listing for an obscure query.
I noticed the patent is not even talking about clicks or actual traffic here. It's talking about how often the document is even chosen for placement in the the search results -- impressions.
In the snippet you quoted I read the word "selected" to mean "clicked", as in selected by the user performing the search. In that context, the bits about changes over time and seasonality could imply weighting for increasing/decreasing popularity.
[edited by: Eathan at 8:46 pm (utc) on June 26, 2007]
So might Google be moving a page back into the Supplemental Index if it just doesn't come up "often enough" for any searches - by whatever threshhold number Google sets?
Just noticed this from the Supplemental Results thread [webmasterworld.com] in the Google Hot Topics section...
2. Why do urls get marked as Supplemental Results?
GoogleGuy's original stated purpose for their Supplemental Index [webmasterworld.com] is "to augment the results for obscure queries."
To quote what GoogleGuy said, back in Aug, 2003, with my emphasis added....
Hey, the supplemental results are a new experimental feature to augment the results for obscure queries. This is a new technology that can return more results for queries that for example have a small number of results. So it might not affect the results for a popular search, but for a researcher doing a more specific query, it can improve the recall of the results. The supplemental collection of pages has been collected from the web just like the 3.3 billion pages in Google's main index.
This conforms with Ted's observation that it's not uncommon to see Supplemental pages returned high up in "obscure searches." Conceivably, Supplemental status might be helping them.
The low impressions thing is easy to disprove. Just make a well-linked page with nothing but fifteen character gibberish words that will never be found for anything. It won't go supplemental if well linked.
" Anyway, a couple quick thoughts:
- This query wasn't new to us at the "Give It Up" session. It's been talked about publicly before, e.g. seobook back in February. If anything, the "Give It Up" mention of this query was more of a reminder to check on why this wasn't fixed yet.
- I believe it's good to remove this query because I don't want people to get fixated on Supplemental Results and focus on them to the exclusion of other aspects of SEO. We saw that happen with the toolbar PageRank bar and ended up slowing the update rate on the visible toolbar PageRank to every 3-4 months so that people didn't spend too much of their time concentrating on PageRank and less on other parts of good SEO.
- Over time, the supplemental results are less and less supplemental and more and more likely to show up for any given query. As I mentioned at SMX Seattle, my personal preference would be to drop the "Supplemental Result" tag altogether because those results are 1) getting fresher and fresher, and 2) starting to show up more and more often for regular web searches. Especially as the supplemental results get more fresh, I'd like to leave that tag behind because it still has some negative connotations for people who remember the previous implementation of supplemental results (which has now mostly been replaced with a newer/better implementation).
Those are the main reasons why I think it makes sense to change this query. "
The theory seems to make good sense, but at what point in time do they start comparing data. It's all well and good for the sites that have pages in the main index, at least their pages have the opportunity to go supplemental.
What about the majority who have many pages already in the supplemental index, will they ever get out, probably not as they will never be clicked on as they tend to be at the back end of the serps. Therefore anybody that is in supplemental hell will be there forevermore.
What about a newish site, will it ever get in the index?
Or will all new pages get in and then filter out?
I think that there are not many searches every month for that keyword but how does that KW make my page go in supplemental? Does google think my page is only worth that KW which is hardly searched?
by the way page I am talking about is from just a month old website.
Is there any minimum benchmark of a KW being searched every day/month i.e. if the number goes below that, your pages related to the KW are in supplemental?
According to Jill Whalen, Dan Crow, director of crawl systems at Google, recently said that "basically the supplemental index is where they put pages that have low PageRank (the real kind) or ones that don’t change very often." (Jill's words)
[edited by: Halfdeck at 2:32 pm (utc) on July 12, 2007]
So might Google be moving a page back into the Supplemental Index if it just doesn't come up "often enough" for any searches - by whatever threshhold number Google sets?
This is interesting, because it kind of bears out what I've seen in recent weeks. My main site has around 650 pages in the index at any given point. The theme of the site has heavily to do with events taking place around this time of year. As we got closer and closer to summer, more and more of my pages came out of supplemental, until now, when I've only got maybe 20 in supplemental (whereas three months ago, I had only 40 that were NOT supplemental) Search queries and traffic have naturally gone WAY up - I'm quite sure that search queries for a great number of those pages spiked in the past couple months - and I'm equally sure that most of those pages are rarely if ever searched for at any other time of year.
So that means my site will probably start falling back into supplemental, as time passes and the queries die down.
