Marcia - 9:31 pm on Aug 1, 2009 (gmt 0) Remember that what was referred to as the "Supplemental Index" retained only the URL and the Document ID.
In some situations, data storage resources may be insufficient...search engine may store "signatures" of documents instead of the (entire) documents themselves to detect changes to document content. In this case, search engine may store a term vector for a document (or page) and monitor it for relatively large changes. According to another implementation, search engine may store and monitor a relatively small portion of the document.
That can relate to indexing of URLs in a partitioned database, which is part of the series of Phrase Based Indexing patents filed. In descending order, partitions allegedly maintain decreasingly less information/data about URLs.
Remember that what was referred to as the "Supplemental Index" retained only the URL and the Document ID.