Welcome to WebmasterWorld Guest from 126.96.36.199
Googlebot turns up and requests my homepage several times during the course of a day. However, when it requests the page, a HTTP code of 404 is frequently being returned. It comes back several hours later, requests the homepage again, but this time a HTTP code of 200 is returned.
There seems to be a loose correlation between the time and the HTTP code returned - 404s seems to be returned early morning (between 6-11), whereas the rest of the time a 200 is returned.
I don't understand why this is happening. As far as I'm aware there has been no server downtime, and even if the server was down, then it wouldn't be possible to record the log.
I've had a look at the homepage through a HTTP viewer and a 200 is always returned. I used Firefox to view the page as Googlebot, and the page is fine.
What's even weirder, is that Slurp appears to have no problems with the site. It requests the homepage during the same time periods as Googlebot, and a 200 is always returned.
Has anyone ever seen anything like this?
You need a 301 redirect to point example.com to www.example.com (or the other way around).
One single inbound link without the "www" subdomain in the URL would create what you're seeing in the logs.
Type http*://yourdomain.com in your browser and see what you get.
browsing the site without the www gave me a 404 error
Yes, that'll be the problem then. Someone has linked to you without the "www" and googlebot is following the link.
Can't help you with a 301 in iis I'm afraid - I suggest you head over to the Website Technology Issues forum to see if you can find out how it's done.
I should just add that this is probably not the reason your site is performing badly in G, although it certainly won't do any harm to fix it and you definitely should - many of your repeat visitors will type in the URL without the WWW and think you've vanished.
Check your own internal link structure while you're about it. Correct any internal links you have to point to the "main" domain (either with or without the WWW sub - whatever you decide to do).
PR passed from www.domain.com/page1.html to www.domain.com/page2.html will be "lost" if for page 2 it is only domain.com/page2.html that is listed in the SERPs.
Use a 301 redirect to fix this. It will help a lot.
Additionally, when you link to an index file inside a folder, make sure that you use only the folder name followed by a trailing / on the URL. Do not include the actual filename of the index file in the link.