Rumbas - 11:19 am on Aug 15, 2002 (gmt 0)

Sorry to hear about your problems with Fast luma. Things have be "fuzzy" lately with reports of irregularities in spidering/updates.

It's very strange that the crawler didn't ask for the robots.txt. It should do that every time it accesses the site. Are you sure?

Actually I haven't noticed the "first page" crawler before. It could be anything from a regular crawler to a special bot they use to check a site out manually (I doubt that - manual checks would be too time consuming).

First page could mean a bot for new sites that they haven't got in the database yet?

Come on guys and girls, help luma out and check if you have the first page crawler in your logs:

"FAST-WebCrawler/3.6/FirstPage (crawler @fast.no; http*//fast.no/support.php? c=faqs/crawler)" "-"

-I can't get the url to the crawler info to resolve either.