- Search Engines
-- Search Engine Spider and User Agent Identification
---- MSNBot posing as Internet Explorer 7 and not reading robots.txt?
- 4:49 pm on Jun 25, 2013
you might recheck your logs and see if these are using the broken UA with the double-trialing spaces.
They are, indeed. Thanks for that, makes it easier to block these requests from my statistics.
Brought to you by