We need to know - from Google - why our rankings disappear

[maisiefantaisie]

As you will see from a couple of my previous posts my homepage has disappeared from SERPS and I am starting to think that I may have been 'sandboxed' by Google.

About 10 days ago our server was infiltrated and started sending out bogus emails. It was immediately shutdown and migrated to a new machine with a new IP address. Some days later our homepage disappeared from SERPS.

Over the years I have read all sorts of posts and queries regarding Google and the sandbox effect. I have never really experienced it myself and now I can see why everybody gets so upset about it.

I'm not bothered too much that we have disappeared from SERPS. I trust that things will straighten themselves out and I have already sent a request for site re-consideration. What is irking me is the silence on the part of Google.

We are a small company and we rely heavily on our Google SERPS and we are very careful to nurture our site always operating within Google guidelines. As far as I can see we've done nothing wrong. It's even possible that the 'sandboxing' is due to a bug in Google.

My problem is that we will never know. Even when it straightens itself out we will never know. And this to me seems somehow wrong. I know that Google needs to protect it's algorithm and can't release too much information, but I really need to know what's going on. It's my business. I need to know if changing the IP affected my business and I need to know if I am or am not in a 'sandbox'. It's quite serious and eventually could lead to me having to lose employees.

Kind regards.

[goodroi]

hi maisiefantaisie,

i am sorry to hear you are having problems. it is never fun to be hacked.

have you reviewed your site to make sure that the hacker did not add hidden text or links?
how much link development do you do every week?
how many pages of new content do you add every week?
how are you monitoring your presence on social media sites?
have you logged into wmt to see if there are any messages?

google does not share specific details about the along but they have given many clues. backlinks, content & social usage data are important.

google also is very careful with protecting their users from possible dangerous websites. since your website was hacked that increase the chance it could be used to damage internet users. what quality signals is your website showing to google?

ps it is dangerous to run a business that is 100% dependent on something you dont have control over. you may want to think about diversifying. do some email marketing, print advertising, social marketing and/or go after bing which now powers about 30% of all internet searches.

[tedster]

There is an unexpected answer to why Google doesn't give specific reasons for ranking changes - no one at Google may even be able to tell!

Search Engine Roundtable has a new article about footer navigation as a possible penalty [seroundtable.com]. The thing that caught my eye here was the reply from Google's JohnMu [google.com] on Google's own forum: "I don't think that's anything that is affecting your site's standing in search results".

Hear that? Even behind the scenes at Google, John doesn't know for sure, and he goes on to recommend "So given that change, I'd let it settle down a bit now..."

Here's what I read into this. Google's ranking process has become incredibly complex and dynamic - it's not the same as just publishing a list. It includes all kinds of statistical modeling for various signals, including dynamic matching of query terms, user intention, traffic types, website content and who knows what.

So ranking changes DO take time to "settle down" when something - almost anything - changes. And in many situations, no one can single out one factor in all this whirlwind and point to it as THE issue.

[martinibuster]

maisiefantaisie, unless you changed something with your site, then it could be something your web host has done, either on purpose or accidentally. There was an incident six years ago where a web host blocked IPs that were grabbing lots of pages. One of them was Googlebots. Here is that discussion [webmasterworld.com]. Possibly a webhost admin may do that to block suspicious IPs, the ones taking many pages at a time. This could be something they do intentionally or by accident.

And here is another discussion [webmasterworld.com], might not be what is at issue, but it's worth reading. It's a case in which a web host using funky redirects killed their client's rankings.

[tedster]

There's some hope here. In a new video, Matt Cutts says that better communication with webmasters is "on the radar" for 2011. [youtube.com...]

[outland88]

You’ve got quite a few things that could be causing problems and they’re not necessarily related to Google. Also check that the new IP is not black listed. Google can bring you back just as quickly if it’s just server problems.

Check how you’re doing in Bing/Yahoo.

Occasionally hosts do block IP’s, for the reasons mentioned, and I’ve caught some in the past year.

Check your log files to see if there are deviations from previous months and to see what the bots are doing.

[bwnbwn]

From my read here is what I see. He shut down the infected machine moved to a new IP so in effect the bot is still hitting the last know IP and getting a 404. Since moving the site was or could not have been done the correct way this maybe your biggest problem. Since it has been done I would sit and wait till the new IP is picked up and your index is back before doing much if anything.

[maisiefantaisie]

goodroi - I've checked for hidden text and links and everything is OK. We don't do too much link development as we are lucky that the site is popular and many people make links to us naturally without request. We don't add new content pages much but we do update pages often, every few months we add some pages. We have a Facebook profile with 3 thousand followers and twitter with 1500. There are no messages in webmaster tools. No crawl problems or duplicate content etc.

As for the website hacking. We are pretty certain it was just the email accounts that were a problem. The server noticed high levels of emails going out and shut everything off pretty quickly. We are still top in Bing for all relevant searches and it hasn't affected these results.

Tedster - Our footer nave hasn't changed for some years, unless Google has changed it's view drastically I think it's unlikely to get de-indexed for this. The Matt Cutts communication about better communication with webmasters sounds great! About time!

martinibuster - I checked with the webhost and nothing has been altered. They put the site on a new machine with a new IP and that's all…oh and they have brand new hardware firewalls. I spoke to another client on the same server and they have seen no problems with their site.

outland88 - I checked and the new IP is not blacklisted and as I said eralier I am doing great in BIng (at the top for all main keywords) We don't rank well in Yahoo, never have…gave up on them years ago! Googlebot always visits the site every few days, today the most recent with 1500 hits.

bwnbwn - I think you might have something here. It could be that Google has not queried DNS and looked for a new IP address, so it would then drop it from index until it finds it again. But if that's the case then Google should get on the case with that problem. It's not fair that we should suffer de-indexing due to their sluggish behaviour.

Thanks everyone for your suggestions. As of today the homepage is still de-indexed. Traffic has fallen considerably. And as far as I can see we still haven't done anything wrong!

[indyank]

Have you tried fetching your home page and a few other popular pages via GWT? Is it normal there?

[indyank]

Tedster, I guess people are reading too much into whatever googlers say and that article on serountable was a bit toooooooooo much...

We never know what John mu saw (he probably saw an entire html sitemap there) and he started of saying that he didn't notice anything in particular...He then added a secondary observation that it was a good idea not to put too many links there (footer)...

Generalizing that for navigational menu and linking them to penalties...uhh...Man, looking at the tweets to that article was really funny.While john mu referred to too many links in the footer, it is now getting spread as "Google Warns Some Navigation Menus Might Be Spam"...huhhh...unbelievable... :)

[maisiefantaisie]

Thanks indyank - Fetch as googlebot is all fine and returning a 200.

Just to let everyone know that Google has promptly dealt with my request for indexing re-consideration and I received this message from them on 23/12/10:

"We've now reviewed your site. When we review a site, we check to see if it's in violation of our Webmaster Guidelines. If we don't find any problems, we'll reconsider our indexing of your site. If your site still doesn't appear in our search results, check our Help Center for steps you can take."

My homepage is still missing from SERPS and after checking the help center for any steps I can take, as far as I can see we seem to be operating within all the guidelines.

This communication from Google is fairly unhelpful and has left me none the wiser. Meanwhile business is teetering. I guess it's not worth requesting a re-consideration again. It feels odd to be left out in the cold when we are a well established company offering a quality service. Please Google, a little more information wouldn't have gone amiss, should I really be expected to figure this out on my own?

[outland88]

Do a few phrase searches from your index page in Google and see what comes up. Do any of the pages of your site appear with the phrase searches? Also for your valued keywords do any of your site’s pages reveal themselves with searches? I’m not talking about in the same position but possibly in the top 100.

Your Yahoo results should mimic those in Bing once they merged.

[maisiefantaisie]

Hi outland88, when i do a site: command search all of my other pages are present except the homepage. When i use text from the homepage in the search, the homepage does not come up.

Though......sometime ago, a few weeks before we disappeared from the index I copied about 30% of the homepage text to an inner page. The reason for this was that Google had started replacing my hompeage in search results for what it thought to be a more relevant page.

For many years it would always show my hompeage, then about 3 months ago it started showing an inner page in it's results. I wanted this page to be more welcoming like my homepage so i duplicated about 30% of the hompeage text on the inner page.

But, i'd be surprised if google would de-list the homepage which has nearly 8000 links incoming (while the inner page has 375) just because of this.

As for Bing, we have been top in relevant search results since it began, but these are not reflected in yahoo. Just checked and results are completely different.

[linux]

ok now about footer links: at germany we have VERY strict laws about what kind of sites a website needs

e.g impress,agb,widerrufrecht(right of withdraw), shipping details... so every site need this informations linked, but where should we set the link (normaly footer)

sorry if google has a problem with this than .... google must die );

[tedster]

That's a VERY widespread kind of footer link, linux - not the kind of spammy thing Google was talking about.

[outland88]

Though......sometime ago, a few weeks before we disappeared from the index I copied about 30% of the homepage text to an inner page. The reason for this was that Google had started replacing my hompeage in search results for what it thought to be a more relevant page.

For many years it would always show my hompeage, then about 3 months ago it started showing an inner page in it's results. I wanted this page to be more welcoming like my homepage so i duplicated about 30% of the hompeage text on the inner page.

I thought a scenario like you mention would emerge and you’re beginning to answer your own questions. I have done the same years back and it took me three months of tinkering to figure it out. I keep BU’s of the changed pages sometimes 2+ years. It is natural to do what you did as a quick fix but sometimes it doesn’t pan out because of the duplication which is perceived as small. Normally I only see the problem when a threshold of about ten pages is reached in Google. Not everybody is penalized due to the size of sites, types of authoring programs, hand coding, and set-up of the site. In other words the complexity of the site can undo the penalty and make it appear as a bug IMO. Remember I’m not looking at your site to say definitely so at best this is a guess.

[maisiefantaisie]

Thanks outland88 - The site has about 250 pages. Only one of the inner pages has duplicate content, we're talking about 60 words. Surely Google wouldn't remove the homepage from the SERPS as a penalty. Seems a bit extreme?

[Robert Charlton]

maisiefantaisie - I can't say that the duplication you mention is responsible for your problem... but I can say that a good many of the sites I've seen over the past several months which have had their home pages disappear have had dupe content issues.

You might also want to search your home page code for a noindex attribute in the robots meta tag (should you have a robots meta tag on the page).

goodroi spotted one of those a few weeks ago in a site review discussion (in the Supporters area), and I've encountered the problem frequently enough that it's one of the first things I check for. In this particular case, there were two head sections on the page, one left over from some recent page revisions, so the tag wasn't where you'd generally look unless you searched for it.