Welcome to WebmasterWorld Guest from 18.104.22.168
Forum Moderators: open
So, as my question was in another thread - did anybody see the same thing that FAST doesnīt go through the hole cycle right now but cut it off?
Looks like as I forgot an update:
with the relaunch last week around the 10th of july the numbers of pages in the index changed a bit. Didnīt notice a spidering between the end of june and the relaunch-update around the 10th.
joined:Oct 23, 2000
22.214.171.124 - - [30/Jul/2001:12:35:47 -0400] "GET /robots.txt HTTP/1.0" 200 134 "-" "FAST-WebCrawler/2.2.6 (firstname.lastname@example.org; [fast.no...]
126.96.36.199 - - [30/Jul/2001:14:00:28 -0400] "GET /DIRECTORY/file.html HTTP/1.0" 304 - "-" "FAST-WebCrawler/2.2.6 (email@example.com; [fast.no...]
BTW anyone know why it apparently starts at a random file?