heini - 9:52 am on Feb 28, 2003 (gmt 0)
Hakre, I'm asking because that appears to be some unusual incident.
First submission did not do much in most cases, apart from triggering a visit for robots txt.
Second a complete adhoc spidering plus full indexing is something I have never seen.
Fast crawlers typically operated similar to Google etc: once they have found some good links they were ready to eventually visit and index a site.