|First-page rankings disappear - many but not all keywords|
I have a site that's been around for over four years. For the past 3+ years I've been first-page for nearly every keyword and phrase I've wanted. Searchers find my site using over 30,000 different keywords and phrases every month. The site gets nearly half a million visitors a month.
Yesterday I noticed that traffic dropped. I did some test searches on Google for phrases for which I've consistently ranked well. They were gone. Not even in the top 500. I continued to search and found that one set of keywords survived intact and the other set did not.
The keywords and phrases that survived were "[insert state name] widget stores." I'm still #1 to #3 for every state in the US for those phrases.
The keywords and phrases that vanished were "Acme widgets," "Company X widgets," etc. I even did a search for link:www.mysite.com/acme_widgets.html and Google came up with no results.
A few weeks ago I added a banner advertiser. In order to get him more traffic I also added a link in my navigation menu that says "widget supplies." The links from his banner and this navigation link are rel=nofollow and go first to a response.redirect page so that I can track the number of clicks he gets.
The pages that vanished all have this "widget supplies" link on them. The state pages do not have this link.
If Google thought I was selling links, wouldn't they ban the entire site? And is what I'm doing even considered selling links?
I've had a few pages go into oblivion before for a couple of weeks and then reappear in the same spot on the first page. But I've never had ALL of my widget pages disappear. In the past I'd do nothing when pages disappeared. Just be patient, make no changes, and they came back. But this one really has me spooked. I lost about 30% of my traffic today because of this.
Has anyone else experienced this? Does anyone have any ideas?
Replies are very much appreciated.
[edited by: tedster at 4:11 am (utc) on Mar. 14, 2008]
The first thing I'd do is go through every page of the SERP to discover whether the urls are completely gone, or perhaps triggered the dreaded -950 penalty [webmasterworld.com]. That one is a real bear and has generated the longest running discussion we've ever had here. (The link I gave goes to the summary.)
The second thing is the banners - you did everything right, even double teamed it by using nofollow and a scripted tracker. If that is really the problem, then a Reconsideration Request would be in order.
Which brings me to one more point. Do you have a Webmaster Tools account? If so, any clues in there? For example, the link reporting is much more complete. And it is the spot to do a Reconsideration request.
(The link: operator results are a bit troubling, possibly indicating a technical glitch or some penalty -- except that link: operator results can be very strange and are always an incomplete sample.)
[edited by: tedster at 7:35 am (utc) on June 2, 2008]
Thanks for the reply, Tedster. Yes, I have a Webmaster Tools account, although the information provided is always dated and sometimes flat out wrong.
I found some pages on different datacenters, albeit far down the list. Others are completely MIA.
The site: operator in Webmaster tools shows the pages are in the index. But that was as of last week. All of these pages disappeared in a single day.
The timing of this couldn't be worse. My $2K a month advertiser will be reviewing the first month's performance on Monday, and I've been working 10-16 hours a day building an online store.
[insert favorite swear word here]
dickbaker i have exactly same problem (and not only me. For some people same thing happened in february and i have read on another forums similar issues for other webmasters). I have response to this thread:
Saturday lose position, Tuesday got back old position even with better results than ever and 12h later lose again.
I have first day request reconsideration because of huge dropping. However because of obviously G game with "lose position for all keywords - back to old position for all keywords" i hope that it was not necessary and that site will be soon back to old positions.
Well, things have changed again, and not for the better.
I did a search for my site's name, which is unique. The home page is only appearing on a couple of datacenters. However, when searching for the site's name, two of my content pages appear in first and second place.
One of them did not get the new link and banner ads added to it (an oversight on my part). The other page has the new link, but not the banner ads. Ironically, this second page is one that hasn't been ranking well.
Wouldn't a ban result in my site not showing for anything?
I wish these were the old days when you could send an email to email@example.com and at least get some kind of answer.
An update for anyone who's interested.
I did a site: search. None of the main pages of the site are in the index, not even the home page. My main category pages are not in the index, nor are the sub-category pages (Acme Widgets, XYZ Company widgets, etc).
Even going one page at a time and searching for site:www.mysite.com/acme_widgets.html shows no results.
The only pages showing in a site: search are those that either do not have the "widget supplies" link in the navigation menu, or those that have cache dates older than March 4th. The latter pages are the individual models of Acme Widgets.
On March 4th I made a very minor change to the page code to correct a defect in the way the page looks. I just added a few <tr> and <td> tags, but the content is still where it was in the source code.
I checked the Google directory today as well. If I search the directory for my site by name, I don't get any results. If I go to the category my site is in, the site is still listed.
I'm out of ideas. If anyone has a suggestion, I'd sure love to hear it.
Yes, it sounds like your domain has been seriously hurt in the Google index - either through a technical glitch (yours or theirs) or through an intentional act by Google. An intentional act seems unlikely given the circumstances that you report. I'd lean toward some kind of technical problems, so definitely check out your source code and be sure everything is still what you intended. Also be sure that no "parasite hosting" has somehow been hacked into your pages, no horrible lines added to your robots.txt file, and so on.
Once you're sure you have no major guidelines violations (and an intentional penalty of this magnitutde would usually take something very egregious) then use your Webmaster Tools account and file a Reconsideration request.
Tedster, thank you for suggesting that I check my robots.txt file and my code. In doing so, I confirmed what my wife has been telling me for years: I'm an idiot.
When I was making those small design changes to correct a defect, I created test templates so that I could make sure the design worked across multiple browsers. I didn't want to run the risk of leaving a test page with duplicate content sitting on the site, so I put "noindex, follow" in the robots tags of the test templates.
I was doing all of this at 2:30 am, and wasn't thinking. When I was satisfied that the new templates worked, I re-named them, removing the "_test" from the template name. But I forgot to change the robots tags to "index."
So, all of the pages I changed have had "noindex, follow" in the robots tag for the last ten days.
I feel like a fool even posting this, but I didn't want to leave people with the impression that there was something unusual going on with Google.
Now my concern is with how long it will take for Google to re-index the pages, and for my pages to regain their former rankings. Weeks? Months? A year?
As Bugs Bunny would say, "what a maroon."
Thanks for letting us know. Mistakes like that are more common than you might imagine. I've got one about a 301 redirect loop that sends chills up my spine. Short description: never redirect your home page to itself!
I'm glad you're going to get your problem fixed.
dickbaker > Thanks for sharing your experience.
I believe that we all do these kind of minor (and rather stupid) mistakes. At least I have done the exact same thing as you did -- forgot to remove the 'noindex' from a newly re-designed home page ;)
Recently, I even forgot to pay my domain name fee for one of my sites -- and for about a week, when visitors and SEs tried to enter the website, they got a 404 - not found :(
In both cases my experiences was that my website regained it's ranking in just a couple of days :)