Forum Moderators: Robert Charlton & goodroi
Run Xenu LinkSleuth over the site to check for problems.
Last week 35 pages have been in index correctly without supplementals. I have focused on content and spent little on link exchange.
Tell us what you did. I 'd like to learn if you have a silver bullet.
>>Imweb, i did none of what you described and yet today most of my pages on one of the subdomains went from supplemental to fully indexed. Go figure...<<
No go figure, please :-)
Tell us which DCs you have noticed that happening on. Thanks.
So this may be a duplicate content issue generated by the www vs. non-www version. And if so, would necessitate some type of 301 work (something I'll need to learn).
Make sense?
For every properly listed site, the alternative URL for the main index page (domain.com vs. www.domain.com) is always shown as a URL-only listing.
I think this is normal behaviour. Google knows that the domain exists, but makes a note that it redirects elsewhere, and then doesn't list any of the internal pages.
For sites where it all goes wrong, Google starts showing a bunch of internal pages from the "wrong version" as supplemental results, and these are impossible to shift.
One way that this happens is if you have both www and non-www indexed (with a whole load of pages filtered out as duplicate content) and you then set up the redirect, what happens is that Google then indexes more and more of the www pages and appears to drop the non-www pages out of the index. If you then change the content of some pages, the non-www pages that had been filtered out as duplicates have their old cache of non-www compared to the more recent cache of www, are found to no longer be duplicate content (the www was updated remember) and so the non-www pages reappear in the index (with Google completely forgetting to check that the pages are actually now just 301 redirects) and have an ancient cache (because no content can be fetched directly from that URL because the URL just redirects elsewhere).
For every properly listed site, the alternative URL for the main index page (domain.com vs. www.domain.com) is always shown as a URL-only listing.
I think this is normal behaviour. Google knows that the domain exists, but makes a note that it redirects elsewhere, and then doesn't list any of the internal pages.
For sites where it all goes wrong, Google starts showing a bunch of internal pages from the "wrong version" as supplemental results, and these are impossible to shift.
One way that this happens is if you have both www and non-www indexed (with a whole load of pages filtered out as duplicate content) and you then set up the redirect, what happens is that Google then indexes more and more of the www pages and appears to drop the non-www pages out of the index. If you then change the content of some pages, the non-www pages that had been filtered out as duplicates have their old cache of non-www compared to the more recent cache of www, are found to no longer be duplicate content (the www was updated remember) and so the non-www pages reappear in the index (with Google completely forgetting to check that the pages are actually now just 301 redirects) and have an ancient cache (because no content can be fetched directly from that URL because the URL just redirects elsewhere). These ghost entries sit in the SERPs for ever more. There is no way to get them removed. Google sees the redirect, but still doesn't drop the ancient cached page for that URL out of the index.
Server Response: [mysite.com...]
HTTP Status Code: HTTP/1.1 200 OK
Date: Wed, 21 Dec 2005 16:17:00 GMT
Server: Apache/1.3.33 (Unix) mod_gzip/1.3.26.1a mod_auth_passthrough/1.8 mod_log_bytes/1.2 mod_bwlimited/1.4 PHP/4.3.11 FrontPage/5.0.2.2635 mod_ssl/2.8.22 OpenSSL/0.9.6b
Last-Modified: Wed, 21 Dec 2005 16:11:41 GMT
ETag: "668801-1ec9-43a97ebd"
Accept-Ranges: bytes
Content-Length: 7881
Connection: close
Content-Type: text/html
Is this the problem?
This question gets asked every day in this forum.
I'm referring to a subdomains (non www) that was indexed by google at one time. Then it all went supplemental after allegrs (if i'm not mistaken) probably because it resembled a link farm too much. I gave up all hope of ever revitalizing it but was still actively exchanging links from it since it was doing very well in MSN and Yahoo. Just yestorday i noticed that it is now back from supplemental and almost fully indexed again. The subdomain never had a 301 issue, dup content or any other problems. It just had more outgoing links then incomong but not anymore. Maybe that is the secret...
Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_HOST} ^domain\.com [NC]
RewriteRule ^(.*)$ http://www.domain.com/$1 [L,R=301]
Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_HOST} ^domain\.com [NC]
RewriteRule ^(.*)$ [domain.com...] [L,R=301]
Edit: I pasted the above into the htaccess file but it hasn't effected the header check info...still giving a 200 without any redirect.
The 301 permanent redirect is in place a week.
On the BigDaddy datacenter, 64.233.179.104 , the site is now indexed correctly and no longer listed as supplemental.
Thanks for your help!
The newest cache date is Dec 17 (it is the index page, the only one that didn't go into the supplemental index).
You need to set the preferences in your FTP program to show all files that begin with a dot to make sure that you are not overwriting an existing file.
