petehall - 2:19 pm on May 24, 2006 (gmt 0)

Being a professional programmer i don't believe in glitches. It is either a bug (too noticeable and going on for too long) or the behavior is by design. Don't underestimate Microsoft's developers. I tent to think the homepage dropping behavior is by design.

So there's no way an error could have occurred during a crawl which won't be rectified until the next crawl?

I think it has something to do with that some sites got a domain.com page and a domain.com/index.htm page, but I think a SE should be able to pig domain.com as the main page, also this seems to take a little longer this time, before it took 1-2 day, but now its has almost been a week. ):

I never link to two versions of the home page and have still had home pages disappear on occasions.

They always re-appear within a short timescale.

I'm sure they probably 'lose' the odd inner page too, however this is more difficult to spot.