heini - 7:11 pm on Jan 24, 2003 (gmt 0)

>It also get most of the links at the end of the root page instead of those important ones in the very beginning.

Same story with a site here. Had initially PFI'ed a couple of selling pages from the site. Now the main crawler has hit the site, took links from a secondary bottom navbar.
An interesting tidbit here is the free pages already appear in the db, but without the usual title and snippet.
Instead ATW gives the filename as title, and the linktext as snippet.