Incorrect title and info in serps, correct cache

[joergnw10]

My host transferred my site to another server, the site was down for some hours and now Google has my homepage indexed with the title

Internal Server Error

and a corresponding snippet. Unfortunately my error page had a title tag in the head!
This happened 11 days ago, the site was cached many times afterwards with the correct title and content, but what is displayed in the serps is incorrect for many (not all) of the important search terms.
Prior to this incident the page had the same title for something like four years. Reading through the forum gave me the impression that this kind of thing should sort itself out within a few days. As I can not detect any change (the searches that result in showing the wrong title don't seem to change or get less) I am now getting worried.
Rankings don't seem to have been affected yet, but with this title I obviously don't get many visitors from Google at the moment. Is there anything I could/ should do?

[tedster]

I would think that time will straighten this out - but you could also use a reconsideration request or post over in Google's webmaster forums to bring some attention to the problem.

[joergnw10]

Thanks very much! Your reply should help to keep me calm for a few days, if nothing changes I will try the routes you suggested.

[TheMadScientist]

Even easier than tedster's suggestion and without worrying about bringing too much attention to it is to simply resubmit the URL to make sure GBot drops by on the next run... It's something I've done before when I'm on a 'wider' crawling schedule and have made a change to a page I want to make sure gets found ASAP or want a page I had been serving a 403 to all bots :) reincluded in the index. I'd just drop the URL in the 'add' box on Google and not even worry about it.

They don't (in my recent experience) crawl pages delivering error status codes nearly as often as they do regular pages and will even basically 'cut off' pages serving error codes from the crawls in some cases. Dropping the URL in the 'add' box on Google seems to trigger an earlier recrawl. (It's not something I've really studied to say for sure, but I've done what I suggested on occasion and it does seem to get the URL recrawled faster.)

When you submit a URL they que it into the crawl, and IMO that's what you need more than anything really.

[edited by: TheMadScientist at 7:15 pm (utc) on Mar 13, 2010]

[tedster]

This also makes me wonder if your server actually returned a correct http status code - rather than just serving up an error message page, but with a 200 OK status.

[TheMadScientist]

Good point tedster...

It could have been a 302 redirect too if somehow there was a custom error page being served. Of course, normally AFAIK 302s don't happen on Internal Server Errors, but I suppose it's possible. I would suggest setting up a test directory and making an .htaccess file error so you can check the actual status code for the error message served and make sure you are getting the correct status code with a header check like the one here in your control panel.

[joergnw10]

I will have a look at my logs tomorrow, guess I should have done that earlier. It might well be what you suspect tedster. The error page was part of the software package and unfortunately that is one of the things I never had a closer look at.
I don't think the crawling is a problem, the Google cache gets renewed almost every day, but for some reason it's only the cache, not the serps.

[joergnw10]

Couldn't really wait until tomorrow... according to the logs there are no errors, just status 200.
What are my chances in this case that this will just 'go away'?

[Robert Charlton]

...according to the logs there are no errors, just status 200.

But if your error page returned a 200 when you were down, that would have created a problem. You want your error page to return a 404.

Try entering a url for a page on your domain that doesn't exist and make sure that it is returning a 404.

Eg...

http://www.example.com/nonexistentpage.html

...should show a 404 status.

That doesn't tell you what your server was returning for your home page when your site was down, but it should give you an idea whether you still have problems.

[joergnw10]

I'm confused. My htaccess file has the following rewrite rule:

ErrorDocument 500 /500.shtml

I don't see any errors in my logs that correspond to the time the site was down. What the serps show is the title and content of the custom 500 error page. I found these instances in the logs that correspond to this:

"GET / HTTP/1.1" 302 - "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

"GET /500.shtml HTTP/1.1" 200 538 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

I don't know what this means exactly, but this probably caused the problem.
The 403 / 404 pages I set up myself and they don't return 200's.
I have deleted the 500 page for now.

[TheMadScientist]

You want your error page to return a 404.

Actually, you want the Error Page to send the correct Error Status Code, which in this case is a 500 Internal Server Error, not a 404. If you really want a custom error page for the 500, then you might try changing it to PHP and serving the header directly from the page to make sure User-Agents get the correct status any time they request the page. Personally, I wouldn't worry about it, because if there is really an internal server error, then your server probably can't serve the custom page, which may be why you were getting the odd status code, but if somehow it does, then you could probably make it work by serving the correct status code from the error page itself, so when a Bot requests the page they would get a 302 redirect (which is recommended against), but would be followed by a 500 Internal Server Error on the landing page, which I would include a noindex meta tag on personally.

[tedster]

Yeah, what he said. It looks like your server (damn that MS) has the default error page set-up for 500 that goes like this:

302 Temporary Redirect >> 200 OK.

That's why your error page got indexed. As long as the redirect is a 302 status, Google will index the content of the target page with the address of the original page. Still, this should all be sorted out quite soon for you. Now you just want to prevent any re-occurrence.

Actually, this set-up might even have saved you some entry traffic. There's a chance your rankings would have vanished completely for a while otherwise, as Google does not want to send traffic to a server that isn't responding. But the minute they got a good response again, things would have straightened out and you would not ever have seen that bad title show up. And with a 500 http status, they would have sent accelerated requests for a while, trying to catch your server's healthy reappearance.

[joergnw10]

Thanks for all your help. I have now set up a php error page that serves the 500 status.
I think what also could have happened is that the server that contains the database with my custom pages (including homepage) could have been down, while the server with the domain was already (or still) working - which is where the custom error pages are located (if that makes sense?).
Fingers crossed this will be sorted soon and not happen again!