| 3:09 pm on Nov 13, 2001 (gmt 0)|
He will be there soon, just wait
| 5:22 pm on Nov 13, 2001 (gmt 0)|
Did you submit the sites, are there links pointing to them?
| 5:30 pm on Nov 13, 2001 (gmt 0)|
I have both submitted the site to the add url page and they have links from existing sites in the fast database.
| 7:21 pm on Nov 14, 2001 (gmt 0)|
HIPO, then it must come very fast. fastbot loves links. I had new sites indexed very recently, just by links, no submitting.
Doublecheck your robot.txt file...
| 12:32 am on Nov 15, 2001 (gmt 0)|
I too am trying to find out why my sites don't get refreshed more often by the FAST webcrawler. This summer they announced that they are refreshing their index every 9-12 days, but even though my sites are well represented in their index, I don't get refreshed very often.
Here's what I've been seeing: a small (test?) crawl in September (9/7-12), a full crawl in late September (9/21-10/3), another crawl (with FAST-WebCrawler/3.3) in mid-October (10/19-26)), and what seems like a full crawl starting yesterday (11/13). So, basically one crawl every 4 weeks.
What is everyone else seeing? Do I have to do something special to be refreshed more often? Should I submit my URLs every 9-12 days?
| 10:01 am on Nov 15, 2001 (gmt 0)|
starec, i was under the impression that fastbot loves links aswell.
all my sites prior to october get spidered on a regular basis, and a great number of these sites have links to my post october sites, but no joy.
we don't use a robot.txt file to exclude any spiders.