Welcome to WebmasterWorld Guest from 188.8.131.52
It left a referring URL. Submitting to LookSmart's paid submit will get you crawled by this bot. Obeys robots.txt. The index is not yet on search.msn.com, but they say they do intend to add it in the future.
It will be interesting to see what they do with it...
Grabbed robots.txt, then index.html, then robots gain, then went deep - all with the same IP. No robots.txt violations.
184.108.40.206 - - [19/Jun/2003:02:05:37 -0400] "GET /robots.txt HTTP/1.1" 200 2507 "-" "MSNBOT/0.1 (http://search.msn.com/msnbot.htm)"
220.127.116.11 - - [19/Jun/2003:02:05:38 -0400] "GET / HTTP/1.1" 200 32464 "-" "MSNBOT/0.1 (http://search.msn.com/msnbot.htm)"
If you don't want MSNBOT grabbing images, PDFs, etc., then you'll need to modify your RewriteRules appropriately. See the discussion in [webmasterworld.com...] about how to do so.
which are all 404s. I've sent them some site logs, but it's happened on various sites and I can't be bothered to be used as a guinea pig for their problems.