Homepage Dropped From Index

Is it a bug or the big stick?

[bakedjake]

PR7 site, competitive industry, site has been up for 5 years. Completely clean - as white hat as can be.

Woke up this morning to find that Google has DROPPED the homepage from the index. I put in the URL, and Google returns a "Sorry, no information is available for the URL" message.

All 150K other pages on the domain continue to remain indexed, however. It is ONLY the homepage that has been dropped.

Anyone else seeing this?

[edited by: bakedjake at 9:36 pm (utc) on Aug. 14, 2004]

[jdMorgan]

wannalearn,

In a nutshell:

Pick one canonical URL for your site, usually www.example.com/ or example.com/, and stick with it.

For internal links and requested incoming links, use that one root URL. Internally, you can use
<a href="/">, <a href="http://www.example.com/"> or <a href="http://example.com/"> -- the first form being "absolute" and the second two being "canonical" -- but always be consistent with the domain name if you use the canonical form.

If your server receives a request for the "wrong" domain name, it should invoke a 301-Moved Permanently redirect to the "correct" domain name. This can be implemented using mod_rewrite on Apache, or with the "control panel" or ISAPI Rewrite on IIS.

The key is to be absolutely consistent in your internal linking, and to try to get those who link to you to use the "correct" domain name as well. Implementing the 301 redirect will help in this respect if the Webmaster adding a link to your site verifies his/her work.

While we have recently had some reports of a specific search engine mishandling 301 redirects, the 301 is still "the right thing to do," and we'll have to leave it to that search provider to bring their redirect handling into conformance with RFC2626 [w3.org].

Personally, I believe it is a mistake for search engines to try too hard to compensate for incorrectly-configured sites, at least insofar as it results in problems for correctly-configured sites.

Anyway, in a nutshell, always link to any given resource by one URL and one URL only; consistency is what matters.

Jim

[creepychris]

It seems Google has fixed this bug. I was hit by it too and the www version of the site was given PR0 and dropped from the index, but now it's back and both versions have their PR = and everything else is equal (backlinks etc).

It also looks like an update is in progress.

[malooga]

I posted earlier about my homepage being dropped, but now the plot thickens. Today it reappeared at #8 for the phrase I have been at #1 for years. Here's the peculiar part.

I use a script to track inbound and outbound hits with my link partners. Sites linking back use the following syntax: [mysite.com...] Well, my site is appearing in G's search results with that URL - including the?username part.

I've been using this script for a long time and my ranking hadn't changed. Now that it has, I'm wondering if my inbound link syntax is a problem. Any thought?

[Stefan]

Sites linking back use the following syntax: [mysite.com...] Well, my site is appearing in G's search results with that URL - including the?username part.

I've been using this script for a long time and my ranking hadn't changed. Now that it has, I'm wondering if my inbound link syntax is a problem.

Yes, it's a problem.

[malooga]

"Yes, it's a problem. "

Since your response was so brief, I'll assume you're referring to the fact that each of my incoming links contains unique text at the end, i.e. the username. O.k., I'm no expert so I'll accept this.

What I don't understand is that I've been using these inbound links for over a year and my home page stayed consistently at #1. Googlebot crawled the page many many times with no effect. And then all of a sudden my ranking changes. If what you say is true, I guess Google must have changed their rules. Wish I was more sure of this before I change everything around ... sigh :-(

[Vork]

My site ranked on top for 2 probably best 3 keyword combos in web development. Today my index page had been removed (now not in top 1000 even!) and this happened for the second time already! The first disappearance happened I believe due to the sloppiness in the internal linkage where links to home where different sitewide (some to company_info.php, some to company_info.html, some absolute ones with http:// some without) After correcting this the home page reappeared only to disappear again in 3 days (after next fresh crawl)!
Can my non-standard filename for index page(not default, index or home) trigger duplicate content penalty of any sort?

I am at my wits' end...

[malooga]

The soap opera continues... This morning my home page reappeared at number one after my site was crawled last night. And now my URL is back to normal - no?username at the end. Guess that wasn't a problem after all. Maybe my site was offline during the previous crawl? Who knows?

[steveb]

malooga you probably have a serious problem. Google is having a hard time deciding what your canonical page is. I see this a lot... some variation of /?blahblah taking over as the primary page due to more link power going to the? url than the root.

You are confusing Google with duplicate content URLs, and that isn't a good idea. if you want to keep the same link style but not risk your main page, have them link to some other entry page like www.domain.com/entry/?blahblah (with somewhat different content than the main page of course).

[Stefan]

Wrt Malooga's problem:

What Steveb said.

I originally started with a longer reply in my post above, but then before hitting "submit" realized it might be a little harsh, so I edited most of it out.

If you feed G multiple URL's for the same page, what do you expect? Alright, it didn't hurt for a year, but apparently one of your new "backlinks" has a high enough PR that it's causing you dupe content problems.

Good luck getting everything changed to your actual URL.

[malooga]

Thanks steveb and Stefan. As I said, I'm not an expert in any of this and didn't realize that the different inbound link url's to my homepage would constitute a problem. As you both seem to have more experience in this than I do, I respect your opinions and will think about a way to get around this. Thanks again.

[Vadim]

question to jdMorgan (msg #:61) and all other who would like to comment.

Pick one canonical URL for your site, usually www.example.com/ or example.com/, and stick with it.

It so happen that I actually followed all the recommendations in your post about a *month* ago.

1.Reduced all links to canonical form in including inbound ones that I knew of.
2. Used 301 redirect.

No effect.

Is it a sandbox?

I mean I did all these because some my pages suddenly went from index and lost PR.

No restoration occurred.

I also changed all java script links for static. No effect.

Is it possible that other factors became more important (inbound and outbound links)?

See more my comments about last in [webmasterworld.com...] (msg 463)

Vadim

[The_Hitcher]

I'm still struggling with a site that has been listed for a year or more in Google and all of a sudden the home page vanishes for keyword searches. Its a clean site. Internal pages respond to keyword searches and it'll come up for a domain search but the home page is not in Google's cache and it shows zero page rank.

In the same month (prior to the index page vanishing act) a competitor totally ripped the home page content and cloaked it thereby producing duplicate content. Since there was a copyright issue we were straight on to them (and their ISP) and got them to remove it. Then we get the index page vanishing act soon after.

Is this indicative of a site about to be dropped or another reason?

[dirkz]

It could be that Google is playing with dupe content filters, or the straight version ("/") and "/?name" are fluctuating in internal PR or relevance due to Google experiments. The higher ranking will be picked, the other nuked as dupe content.

[Nuttakorn]

"Google doesn't want a few sites from the same owner appear at the same time in the SERPs for a search"

I would ask you more details about this. I have create many domains for hotel reservations so I have seperated it in each city for a domain. One City One Domain. Is this allowed by Google? The content is unique in each website. But I have crossed link together. I have use the same host, some it same ip address. Same Whois information.