bobriggs - 12:57 am on Mar 17, 2002 (gmt 0)

Good luck mattdwells.

I also had this observation: My site uses SSI, and all of the pages that do not change very often use the X-Bit Hack method. These are the only pages included in this index on my site. Other pages, which are 2 levels down in the site structure, do not return the last-modified date header. Some pages which are on the same level, are included in the index, and they return a last-modified header.

Can I assume that if the pages are a certain number of levels down in the subdirectory structure AND do not return a last modified date/time that they are not included in the index?

The only reason I'm making this assumption is because webmasterworld.com does not show a last modified date in the cache page, but does show the last date spidered.