Forum Moderators: open
Is there a cycle to this?
Cheers
the site was found using the directory search but wasn't listed for a couple of months, and then in the last 2/3 weeks it appeared in the directory too.
download the RDF file (content.rdf.u8.gz) and grep for the URL
Keep in mind that this is a huge file ;-)
The last time I downloaded that dump was in March and it was then 224 MB (1.15 GB unzipped!).
Needless to say you need some programs to split this file in manageable chunks.
WBF
Updated category pages are appearing intermittently on the public servers -- from one page I was editing earlier I can now see "Last update: 14:15 PT, Thursday, September 4, 2003"
Note that there are now multiple public servers, so you (and googlebot) may see current pages one moment, and months-old pages the next. I don't know how long this stage of the process will last.
So in other words there is no idea about any kind of time frame?
SN
Emergency-priority ODP issues, and non-ODP issues, may intrude on whatever work needs to be done on this.
"All three frontend servers have now been updated. Their data should never be more than a week out of date, compared to editors.dmoz.org. Pages will normally be refreshed when they're more than four days old."
My Targum claims the new process also automatically handles the various "official mirrors."
Problems? Exceptions? Report them in the public ODP Bugs and Features forum, or (a more chancy approach) sticky-mail any frequent visitor who is an ODP editor here.
the category date on dmoz.org is:
Last update: 20:48 PT, Wednesday, July 9, 2003
the category date on editors.dmoz.org is:
Last update: 12:21 PT, Tuesday, August 26, 2003
but the page shows in neither one. It's been coming up in the searches long before the 26th of August.
When will the searches move to the categories?
SN
Search indexes will be built, all at once, just like they always have been, from the RDF, which is cut weekly, all at once, from the working database. The RDF is always at least 3-10 days out of date.
Categories on the public pages are built "on the fly" from the working database, base on this new scheme. They are never supposed to be more than four days out of date. (The curious coincidence of the times involved may lead to the theory that they are built from the RDF; that theory I can confidently deny, since this week I have seen some public pages that were only minutes out of date.)
SN
