Forum Moderators: open
Got hit by:
193.205.4.11 - - [11/Nov/2002:08:54:55 -0700] "GET /pages/ HTTP/1.0" 403 746 "-" "parallelContextFocusCrawler1.1parallelContextFocusCrawler1.1"
It checked robots.txt, but did NOT follow it. Hit rate of 2-3 p/sec... no pauses.
dave