Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

X-Google-Crawl-Date -- why are these pages in the index?

Pages with X-Google-Crawl-Date: in the title

         

whatcartridge

10:15 am on Apr 17, 2006 (gmt 0)

10+ Year Member



Recently I did a thorough check of one of my sites in Google's index and came up with a page that had X-Google-Crawl-Date: (with date and time) as the title. The page was blank except for one line repeating the title.

When I searched on the phrase "X-Google-Crawl-Date:" there were over 6 million results! They were all old - from November and December 2004. Most were dead sites but some were still active. One of the pages was news of the wierds FAQ page with a PR of 6, and other PR4 pages were there too!

I can't understand 3 things:

How does the Googlebot end up seeing a page that way; and

Why haven't they been recrawled and re-indexed (some of them are valid index pages of websites) or dropped; and

Why the hell are these pages in the index?

Is this a serious problem for Google? It is for the webmasters.

g1smd

5:46 pm on Apr 17, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I have never seen that before, but thanks for pointing it out.

Seems like some sort of bug at Google's end.

whatcartridge

6:18 am on Apr 18, 2006 (gmt 0)

10+ Year Member



I just checked again and now I have 32 such pages from one of my sites in the index. Some have been banned from SEs from day one in robots.txt