How may front pages did you lose?

Google kicks clean html front pages

[stinkfoot]

I lost 2 top ranking front pages
All other pages on the sites still getting a few hits
both still top ranked by msn
urls = keyword1 - keyword2 - keyword3 - keyword4

Anyone else with losses please post like to get an idea of simliarities

[layer8]

Hi all,

There has been a lot of discussions about these problems, Last months roling index was in mid stage when my ISP had big problems.

I contacted them and said I was worried about my pages being lost in SERPS, last months changes did not get updated except for a few keyword terms. (These were updated before the problem).

About 5-6 days ago, I noticed the pages that had not got updated last month vanished from the SERPS. The only ones still in there were from the ones updated last month.

I contacted Google and they assured me that I had not been penalized and that the 'temporary' problem should be resolved in next update.

This is bad timing as not much trade gets done in December for my industry and so the last quater of 2003 have not been good for search engine marketing.

This could be connected to your problems or not, I'm not too sure, It could even be a problem on Google's side as well. Obviously they would not tell you this as it is not in anyones interest. I would say somewhere something has definately gone wrong, the only thing I can suggest is wait patiently - I have already started building a large database of reciprocal links, not to boost my search engine rankings but to build more webs to catch my prey.

If you rely on Google then you can only get burnt if things go wrong, this might not even of been a human error. Technology does go wrong, (at the worst possible time in my experience). The thing that comes to mind is do not have all your eggs in one basket.

[Stefan]

Wouldn't the duplicate content penalty also apply to www.domain.com/widgets.html and domain.com/widgets.html?

Yep. If googlebot comes in on domain.com, then follows links, it can do it to more than the index. Steveb was saying Google got better at figuring it out post dom/esm... maybe they just suddenly got less better at it. Either way, it nailed us in dom/esm... why take chances? It's something that's fairly easy to take care of.

[Iluvlabs]

"Try this, iluvlabs.
If a search for "page title" for the index used to show up on the first page, try it again, eg "widgets r us", then add onto the URL of the Google search page, in the address bar, "&filter=0" then hit it again. This can show the duplicate domain.com and www.domain.com pages."

I tried that, at least I thought I did and my site did not show up. I put my sites name in the google search box and then I put &filter=0 in the address box and then I hit search web. Then I tried it and I hit go.

[Stefan]

You need to do the search first, then add &filter=0 to the google serp url, then enter that. I edited my previous post to you... it might not help. It found someone who was linking to us as www.oursite.org/?theirsite.com. That one was a pain... it took Ink about 5 months to get it straightened out, google took a few weeks.

[caveman]

I suppose this is something I should have taken care of long ago, but didn't realize it was a significant issue with G...

Does any one have the definitive experience or latest thinking on best way to redirect on Apache? (I'm referring to virtual server versus .htaccess...)

I've heard some say not to use FollowSymLinks method.

TIA

PS I can't believe this is causing all the lost index page problems...

[rainborick]

The main page on my primary income site lost its top five ranking for two search terms that fortunately were more of a vanity issue for me than one of revenue, but I have four very friendly competitors with whom I had been sharing these positions who also lost their rankings. I promise you that (a) I wasn't doing anything beyond the pale because I'm too much of a coward and this is our money site, and (b) my competitors have no inkling of search engine optimization. They built their sites themselves and do not use any consultants. I've known these guys for 15 years. They're not up to it either technically *or* ethically to try to run any kind of game on Google.

It seems to me that whatever the combination of factors that is now causing these changes in the SERPs, woe betide the unwitting webmaster who happens to have a company or website named "Joe's Blue Widgets" and uses those terms in the <title>, <h1> tag, and/or anchor text, because your main page (and likely the rest of the site as well) is going to be invisible in Google for any searches on "blue widgets."

I just recently started to help a site and began by what I thought was simply cleaning up the HTML and emphasizing their site name which happened to fall into this pattern that included a product name. I just wanted to establish a baseline for things by pulling out the triply-nested <table>s, the flurry of <font> tags, and bloated JavaScript left behind by Dreamweaver. They had been ranking about #185 for their single broad keyword "widgets". Post-Florida they are now totally unranked for that term.

I had a woman call me the other night who had seen my posts online and eventually tracked me down because she was so desparate and upset. She has an excellent little site that followed the guidelines on www.google.com/webmasters/ in every way, but fell afoul of this new deadly combination in her main page by simply trying to do better.

These folks don't know from mirrors, spam, and cloaking and the other deceptive ploys we speak of here. They're just trying to compete fairly with older, bigger sites with bigger budgets. None of them did anything that wasn't in #93 of Google Hacks - a book that includes a enthusiastic message in the forward by "the Google Engineering Team."

If Google is now considering that webmasters who include the names of their products in the <title>, <h> tags, and/or anchor text for a site's main page are violating the "do things for users and not search engines" principle, they should spell it out more clearly in the guidelines so that folks like these can rely on them. I understand that Google can't post a handbook to optimization for all and sundry, but if this is an anti-spam effort it is taking out too many innocents, as well as seriously damaging the search quality.

[Stefan]

site.com redirect

This addresses the problem, courtesy of Yidaki:

[webmasterworld.com...]

msg #2

[caveman]

Stefan,

I tried that once before and again just now and both times received a 500 server error. Any suggestions?

thanks
caveman

[Stefan]

You tried Yidaki's redirect code? I'm not the one to ask really... I took care of it by having the incoming links changed.

Why don't you bump that thread up? If this is really a source of problems for some of the missing index pages, it should get moved up to where some of those here who are computer geniuses will spot it and give some good advice.

[Lorel]

I manage about 15 sites. Most of them haven't changed, a few have increased hits in the last few weeks.

However one has dissapeared entirely. It's a non-profit site that hasn't changed much in the last 2 years since it went online--had a PR of 4. It is still in the google directory (and ODP) but even if I go to that page in the google directory and search for the name of that site right on that page I get a no response--doesn't matter if I use www.domain.com or domain.com or just the name of the site--no show.

I first noticed a problem on the 21st when there were no more referral links in the site meter (still hits from other engines), then I started checking Google every day to see if it had come back.

I always follow google rules for designing a site so I don't think that's a problem.

Any suggestions?

[Stefan]

lorel, perhaps it just got lost in the shuffle...

You could try adding a fresh page, or three, to the missing site, give a fresh link into a new page, from one of your other sites that gets hit by the bot often, and see if Google finds it.

[Powdork]

How would the problem with the redirect occur for some keyphrases but not for others?

[Stefan]

I don't know, Powdork. Good question.

Whenever I hear, "missing index", that's what I always suspect to be the problem, much like Brett did in his post earlier in the thread, because we got hit by it in may/june, (not excatly sure of the time-frame now). Maybe it's not a source this time, but I always figure, best to be methodical and take care of the simple things first... especially if you don't know what the hec is really going on.

[steveb]

A "redirect problem" will not effect keyphrases, it will effect sites.

[Powdork]

Here is where I am confused. If I am redirecting from domain.com to www.domain.com, do I place the redirect in the .htaccess file of www?

Okay, lets walk over to the other side of the room.
Now, Here is where I am confused.

It occurs to me that no matter where I am, it is where I am confused.;)

[Brett_Tabke]

Wow, lots of examples rolled in today. You can stop now ;-)

I'll see how many I can wade through in the morning. We'll see what some of those are showing. All were emphatic that there wasn't a www.domain.com vs a domain.com problem.

Certainly agrred that - either way, Google should be seeing the difference between the two and realizing that www is the preferred method - which is not necc the one that will have the higher pr.

[Stefan]

If you read Brett's first msg, it indicates actual duplicate content that occurs after having incoming links in the wrong form.

I see plenty of www index pages doing just fine without a redirect on domain.com.

You don't need the redirect unless those links are out there. The kw change for serps, Powdork, I don't know.

This is my theory:

The index/default page is where the problem will first manifest itself because of an incoming link, domain.com. This is the page that most webmasters probably check regularly compared to deep inside pages. It ususally gets spotted early and they scramble around at places like this, maybe get it figured out, fixed, and all is good. But if...

Wouldn't the duplicate content penalty also apply to www.domain.com/widgets.html and domain.com/widgets.html?

If the index problem, where it starts, doesn't get spotted by the webmaster, it can eventually propagate through the entire site. The bot comes in on domain.com, then goes to domain.com/next, and will eventually find duplicate pages right through. Google sticks with the older versions for a while before it decides the whole mess is final, then... that's all she wrote.

I might just be blowing smoke out my hat on this, but there is a slight possibility that I'm right.

<edit>Brett posted a second message while I was typing out the above msg</edit>

[Powdork]

A "redirect problem" will not effect keyphrases, it will effect sites.

I agree.
So why is Brett (& GG) feeding us this

I've looked at over 15 "lost index pages". All 15 were webmaster fault.

or this

If you survive in the top 10 under any keyword for more than 30 days - count your lucky stars, as there is a search engine that has just made a mistake.

Something is rotten in Denmark

and while I was typing, this

No no no no. There is ZERO evidence that many of the pages that took hits in this update were quality. Far far far from the truth.

[steveb]

Just a little tidbit following up on message 23. A few minutes ago the newest/freshest version of one of the pages I'm trying to get rid of entered the index. Since I removed all the instances of keyword from the page, and it is only now ranking based on anchor text that has not existed for about three weeks, it plummeted from #7 to #26 and #12 to #55 for two variations on its topic.

But then when I do a search to exclude this page by adding -page1.html, my new page appears ranked at #9 and #20 -- *higher* than the other page. I assume this will sort itself out in a day or so, but that's kinda funny for a lower ranked page to prevent a higher ranked one from appearing.

Which makes me wonder... perhaps if people "missing" an index page but having another subpage in the serps for that term would exclude the page being shown will that make the index page appear?

[caveman]

I'll bump up that other thread as Stefan suggested so we can get input from the experts in here on the preferred method these days for permanently directing traffic from [site.html...] to [site.html....]

But clearly it's not logical that this would be an issue confined mainy to commercial sites ... unless all the non-commercial webmasters are universally more clever and well informed than the commercial ones.

Well...nevermind. ;-)

[Stefan]

Which makes me wonder... perhaps if people "missing" an index page but having another subpage in the serps for that term would exclude the page being shown will that make the index page appear?

Good thought. That's maybe a good way to dig out a duped/buried page.

I checked a missing index page for someone earlier that turned out to not be missing, just buried... I tried a search on a unique piece of text, and there it was.

[pali]

I guess I was lucky.
a)I use to be #12 for kw-kw-kw-kw now I am #17 "BUT" everybody around me is GONE. Completely - zilch: so, it's kind ok with me. All new pages took their places.

b)On my other kw-kw-kw-kw word I am gone with all the others, "BUT" I am dominating top five places with kw-kw-kw.

on the a) situation is a complete mess. First position has one kw on
the page, but second guy has like 20 kw. I am lost and waiting to see what happens. I don't think their no point to make any changes at this time yet.

[Spica]

Unsuspected source of duplicate content: cached copy of your site posted as a page in a web directory!

If a penalty for duplicate content is the reason why so many clean sites have disappeared from the SERPS for their major keywords, then I have just found out a possible reason why mine is in this group. Obviously, a lot of other people would be similarly affected.

I discovered this duplicate page by doing a search with "my keywords -oisfiojsd", which brings back my site at its usual #3 position. At position # 149, I was very surprised to find a page that is a cached copy of my site (without the graphics, but with all the links active). This page is on a directory site. They apparently added my site to their listings last month. Next to each listing, they add a link to a cached copy of the listed sites! These pages are in the form:
nameofthisdirectory.com/common/cache/##/###/#####.asp

So, now, there is another web page indexed by Google, which has the same title as my index page, and links to me! Is this the cause of my problem?

I'll sticky the name of this directory if anyone wants to know. Are other directories doing similar things?

Should I send a spam report to Google?

[GoogleGuy]

Hey, about the redirect--Google tries to find the canonical version of a page, but technically in html I think that all of
domain.com
www.domain.com
domain.com/
www.domain.com/
domain.com/index.html
www.domain.com/index.html

can be different pages (not sure about the trailing slash, but I think so). So Google tries to figure out the best page to call canonical, but it never hurts to give a spider extra help. If you want www.domain.com to be the canonical, you can make domain.com and www.domain.com/index.html be a permanent redirect to www.domain.com. I remember a long time ago (months?) someone had moved to a new domain, and olddomain.com did a 301 to newdomain.com, then newdomain.com did a 301 to newdomain.com/index.html, and so on. Instead of creating a long chain like that, I'd pick one page you want to be canonical (I'd recommend www.domain.com) and make aliases that you don't want be a 301 to that canonical page. As steveb mentioned we normally get it right, esp. compared to the infrastructure changes of a few months ago, but it never hurts to give a spider extra help.

Let's see. Bek, nobody has an "in" with Google, although you wouldn't believe the number of folks who try to claim that. :) I checked out the site you mentioned, but many boxes were checked (Hidden text or links, Deceptive redirects, Doorway pages, and Duplicate site or pages), and I didn't see that at first glance. So it's possible that someone checked it out and didn't see what you reported. I saw some words in the noframes section, but that's it. Could you say in a spamreport what exactly was wrong with the site?

[Powdork]

I was just messing around and trying to find out what would happen if I added nonsense words before the phrases so i tried 'snoopy tahoe weddings'. rfgdxm1 you'll be glad to know that a mirror of your bookmarks comes in first, then your bookmarks. However, your bookmarks are listed as newhoo.com/bookmarks....
Also, first I tried 'sfddgf tahoe weddings' and it said there were no pages that contained 'all the terms'. Has that always been default?

[coosblues]

I lament with all of you that have lost your index page. Shortly after the update my index page dropped from its' 3rd position to #93, but about two days ago as I was watching the datacenters it started making a come-back. Today, it's back in the #3 spot across all datacenters. There were times I would check my page and Google showed it as mydomain.com rather than www.mydomain.com but that seems to be resolved - at least for my web site. I suppose I am one of the very few that has a completely non-commercial web site so the initial loss was only traffic, not money. I might add that every link within and to my site uses the full url with the www.mydomain.com - none use just mydomain.com. HTH

[squared]

Alright. Here's my situation:

Informational site with name and url structure of www.keyword1-keyword2-keyword3.tld. Lots and lots of quality content.

Site launched six months ago.

Doing great until last Saturday. Lots of links from .edus and .govs.

Did get to first page for main keyword phrase, but now on page 7. All other pages are doing great.

Results in my sector do look worse than previously. Pretty decent relative to other engines, but tend to fall off after the second page. Also, noticeably more pages in SERPS with only vague connection between search query.

Other "branded" sites with keywords in incoming anchor text tended to hold their positions for main keyword.

I'm in the educational/commercial area.

I don't think these SERPS are gonna stick. I just wanna know when is it gonna change?

[beadguys]

Once again, I get to be one of the unlucky ones. My index page ranked top 5 for 10-12 of my top terms until Florida. Now, all but one term is ranking below 30.

I run a clean ship, I took over our site last year and made our site spic and span using the info from this forum and the Google guidelines. Being a site with a shopping cart, I've been limited to the pages that the bot can see, so I've had to rely heavily on the index page.

If Google is going to dump index pages, how do I get back into the rankings for generalized search terms. For example, in my industry, the top search term is "wholesale widgets", with widgets being the general type of item we sell. My index page tells people that we have lots of different types of widgets and we sell them wholesale. It lists the names of some of the more popular widgets, and it has links to the shopping cart pages where they can find and buy these widgets.

So, why does Google now consider this page to be garbage? I can't think of a more honest way to create an index page with the purpose of targeting the people who are looking for your products.

An earlier post in this thread suggests removing the search terms from the page. What? So, if I remove the term "wholesale widgets", my ranking will improve? You are going to have to explain that one to me.

Jacqueline

[Nicola]

>>I suspect that for every mega-spammer who has been inconvenienced, there are four or five mom-and-pop enterprises that are in danger of going out of business. You may like the results today, but you could be targeted tomorrow.
--------------------------------------------

Not gonna be a very nice Christmas for alot of people who ran clean relevant sites with good content!

I hope the algo-smiths at Google sleep well over the Christmas peroid, knowing they have put alot of people out of business!

Sorry if this statement sounds strong, but it's the truth.

[jasonale]

I too have a site that has been at the top of google for several different good keywords and the index page has fallen off the face of google. it would be great if someone can fill us in why.