- - Search Engines
- -- Search Engine Spider and User Agent Identification
- ---- MJ12bot
wilderness - 5:55 pm on Jun 21, 2009 (gmt 0)
|Point is, I'm simply reporting another bot that hits many of my sites and the vast majority of the time does not request robots.txt. |
In spite of my stated objectivity and courtesy?
I've had this bot denied from my sites since its inception.
Thread source: http://www.webmasterworld.com/search_engine_spiders/3937580.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com