Google not crawling site any more due to 404s.

[kila_m]

Googles been indexing my site for a long time but during big daddy its not crawling and has 0 pages in its index.

At first I thought my site had been banned but looking at sitemaps it appears its because its a PHP site and I have "index.htm" the main code redirect to:

<?php
header('location: [#*$!xxxx.co.uk...]
?>

Google is no longer going to the correct place and is giving 404 errors.

Any ideas?

[tedster]

In my opinion, it is never a good idea not to resolve the domain root directly -- that is, never redirect the pure domain name. I have seen problem after problem with sites that do this. My advice is always to return content and a 200 header for www.example.com. You can take the internal navigation to whatever url structure you need from there, but give Google a home page.

[JeffOstroff]

Also, take the hint from Google's sitemaps program online. They like to see real 404 error pages, not re-directs.

We had a redirect that sent all error URLs on our site to the main page, but Google does not like that. So when in Rome...

Sitempas would not let us get past the verification stage until a random error URL which they generate produces a true 404 result instead of a 301 or 302 or 200 result.

Very strange requirement, but if you look at all the big names like CNN, Google, Yahoo, and try a bogus URL on their site, you get a real 404 error page. That's the preferred method of having your house in order according Google.

So moral of the sotry, use a header checker tool, and verify your home page spits out only a 200 code, and that your errorn URLs all spit out a 404 code, and pop up an error page, preferably with navigation on it.

[tedster]

They like to see real 404 error pages...

Absolutely. Check those server headers to see what is REALLY being sent.