Forum Moderators: Robert Charlton & goodroi
From a few brief searches in areas I know I am pretty sure that other sites have been given the same treatment so I am a little surprised that there is no talk of it here yet.
I am wondering if other people are seeing this or if it is just my site that has got lucky.
[edited by: tedster at 6:31 pm (utc) on Jan. 7, 2008]
[edit reason] split from another thread [/edit]
My two main sites now have 10 times as many pages in the main index as they did a week ago. I have done a lot of work cleaning one of these sites up in the last month, so whether it is down to that or some other factor I can't be sure.
Hope it sticks! I'm in the middle of a ton of work to sort the non-inclusion, but it all helps.
I have seen one search that had 9 different results in the top 10 depending on which datacentre you were looking.
Paul
At the moment I am seeing the newer mixed results at most datacentres, earlier today when I was seeing the differences I was generally comparing the old results on 64.233.161.107 with the new 'mixed' results on 64.233.163.107.
Obviously I am only seeing changes on typical long tail searches rather than major search terms.
For the record, I see no signs of any change in the SERPS.
If it's true that huge numbers of pages have moved out of supplemental and into the top SERPS on a massive scale
This is not necessarily what's going on. According to Google, the Supplemental results are now available for every search rather than just some searches. Also note that olias reports "Obviously I am only seeing changes on typical long tail searches rather than major search terms."
[edited by: tedster at 4:31 pm (utc) on April 7, 2008]
The types of searches where I am seeing changes are of the form Business Name Place Name.
Before the supplemental index came along these searches would return pages about the individual business from a number of related business directories.
After the supplemental index was implemented and had swallowed up most of the business specific pages from most of the sites what we would tend to see was the SERPS returning the pages from these sites about the place name in question, but not the individual business page (except when there were less that 1,000 results and the supps got included).
Now I am seeing SERPS that look exactly like they did 4 years ago across many datacentres. I did wonder if there may be something as simple as raising the barrier at which supplementals are called up to the SERPS, but I have seen searches with upto 500,000 results that are now pulling in what were my supplemental pages into 1st place - followed by a number of the other similar sites out there.
I don't think the inclusion of my extra pages will affect any major search terms, just the long tail ones in my area (which is not overly competetive)
There are several datacentres with very different results, and some datacentres have results numbers that are very different.
I am not sure that I would go so far as to say that these sup pages have actually moved into the main index - for me, the distinction between the main and supplemental indexes has blurred - the /* indicator tells me that I still have only 3,000 pages in the main index; but when I search for specific text strings on pages that i am sure are in the sup index, they rank well in the serp and there is no "supplemental result" green text (has that been dropped?).
Whatever the case, any move on google's part to broaden the index is a positive one.
So far this is just another way of Google rearranging the mess in their room. No progress in general, but some pages rescued, with no rhyme or reason for the difference.
There are still two indexes though.
Most URLs are still buried in the supplemental hell. Their index is quite small when you think they still can't make the supplemental URLs searchable. I still believe 80%-90% URLs in Googles index consist of supplemental URLs and therefore in most cases are not searchable.
The title of this thread is misleading, it should be
"Supplemental URLs still not ranking in Google".
[edited by: SEOPTI at 11:46 pm (utc) on Feb. 5, 2008]
If I get a few extra pages in the main index, or vice versa, the traffic rises and falls measurably. For Google (or anyone) to tell me that the supplemental pages are ranking/showing, or being searched more, is frankly just a plain insult to my intelligence.
Further, I can honestly say that my website has some of the most up to date, pertinent and authoritative information in my field... yet it sits there languishing in some old supplemental hard drive where nobody can read it... what a loss for the web user in general... pffft (snort of incredulity).
[edited by: Asia_Expat at 10:14 am (utc) on Feb. 6, 2008]
Now supplementals often have a cache displayed, but it is a phantom... the pages will not rank at all for any long bits of text swiped from the page.
Pages in the supplemental index, without a parallel result in the main index, now often have their caches ignored and are not allowed to rank for anything.
The thread title is nonsense. Supplemental URLs are now worse off than they ever have been... and that fact is deceptively hidden by Google by showing phantom caches for non-indexed pages.
My co-moderator Robert_Charlton recently shared an interesting case where the supplemental url is returned -- but only for very on-topic search phrases and not for just any unique string taken form the page. It's as if parts of the page content are tagged as "yes" or "no" when it comes to search retrieval.
I plan on studying this a bit more and try to find other examples of when a supplemental url might rank today. It's certainly true that everything in Google's cached page for a supplemental is not retrievable through a search.
I will edit the title of this thread to read "Some Supplemental URLs..."
...an interesting case where the supplemental url is returned -- but only for very on-topic search phrases and not for just any unique string taken form the page. It's as if parts of the page content are tagged as "yes" or "no" when it comes to search retrieval....
Some examplified discussion about what I was seeing is in this thread, where the poster thought that content in his headings wasn't being returned because they were h2 rather than h1....
h2 content not ranking - versus page content
[webmasterworld.com...]
As it turned out, what wasn't being returned, as far as I can tell, was anything that wasn't semantically related to the "theme," to use an old familiar word, that Google had assigned to the site.
Assuming, say, a car-related site, this is my take on what was going on....
...if you search for an exactly quoted passage unique to your page that contains a word like "steering wheel" or "rear windshield" or "convertible," Google is apparently assuming that your car-related site might have something relevant to return on a car-related search, however obscure, and it appears to return that passage.It isn't returning searches for non-car-related passages.
To me, this is something new that I haven't seen Google do before, but it's not unlike the way Google treats dupe content. If you have enough links to support you, Google will rank it. I've seen pages that have been scraped enough temporarilly fall out for a 10-word quoted passage while still ranking on a competitive three-word phrase that's supported by a title and inbound linking.
The difference now is that Google is doing this for very obscure quoted passages containing semantically-related words, as opposed to exact vocabulary in your inbound links or title.
I'm thinking this has lots of implications beyond supplementals, suggesting the way Google is beginning to look at everything.
To coin a phrase here, you might describe search in part as "relevant differentiation." With no differentiation on the page level... too much similarity and not enough inbound link differentiation.. titles all the same... not enough PageRank to suggest crawl or indexing value... whatever, Google needs to stop looking (or to stop computations in its database) because there's a big use of resources involved. Without differentiation, it's all dupe content.
But, given enough clues, Google really wants to return results.
I think... and this is conjecture from someone who's not an engineer or a data retrieval expert... that Google is beginning to apply semantic classification on many levels, and it may be that some disambiguation of supplementals is one of them that's coming.
[edited by: Robert_Charlton at 12:30 am (utc) on Feb. 8, 2008]
Once again, Google succeeds in doing something so goofy no one could expect it.
As it turned out, what wasn't being returned, as far as I can tell, was anything that wasn't semantically related to the "theme," to use an old familiar word, that Google had assigned to the site.
Well, this was a nice theory, and it may in fact have been correct when I tried it, but I'm seeing some test results change over time, indicating perhaps different results at different data centers. There are enough exceptions and possibilities now that I would draw no conclusions.
Your thoughts?
I was reading a blog post today from a respected SEO blogger who was discussing something similar to get pages out of the supplemental index.I understand the purpose of the link tag “nofollow”. I only use it on my site for outbound links when needed, however; if I want to block Google from certain non-content pages (Legal Notice ¦ Privacy Policy ¦ Terms of Use ¦ Contact Us…) I do that via a deny in the robots.txt file.
I found one of the reader comments on this blog post interesting:
In the context of trying to flow PR to your true content pages in an effort to help to get them out of the supplemental index, and not to waste PR on the way to your real content pages; it was suggested (unlike the “nofollow” link tag mentioned in the posts above) to use the meta: <meta name="Robots" content="follow, noindex"> on ALL pages (category navigation, tag clouds, sitemaps…) with the exception of your index page and your REAL content pages.
Which, as it was suggested (or as I understood it), would accomplish two things:
1) It would still give you the benefit of the onsite navigation PR flow that is followed on the noindex-ed pages, however (as it was suggested) that by not indexing those pages, it would flow more PR to your real content pages.
2) And leave room in the main index for more of your site to get bailed out of the supplemental index.
Has anyone tried this? Results?
