Welcome to WebmasterWorld Guest from 126.96.36.199
Forum Moderators: open
joined:Oct 23, 2000
On the other hand I see some pages that I know are not PFI get respidered on a rather frequently basis.
Like said before, FAST does not spider the site at once. Last month I renamed a few pages and ATW lists both old and new file names. I think this will corrected once the spider tries to index old pages. I believe also that they don't do a massive update like Google. As they get new pages indexed they add it almost "live".
using the URL Investigator I can read about "Last changed". (for my homepage 2003-06-09).
But this date is valid for the Homepage only, not for all other pages.
Last time FAST-Webcrawler visited parts of my site, was 2003-05-06.
After then FAST-Webcraler/3.7./Firstpage visited regularly (every 1-4 days) the homepage (index.shtml) only.
(date format: yyyy-mm-dd)
[edited by: h_b_k at 12:03 pm (utc) on June 11, 2003]
is this depending on the "importance" of a page?
or is 1,5 month today the regular behavior of fast?
time before I had a shorter cycle of deepcrawls.