adamseabrook - 9:17 pm on Sep 20, 2013 (gmt 0)
Sorry my response had a typo in it and was not well worded. What I was trying to say was if you have two directives specific to meanpathbot with conflicting disallow and allow statements we take the first one found. We find multiple conflicting directives often especially on sites which have dynamic bot blockers that add things to robots.txt on the fly. Meanpathbot should act the same as he Googlebot so if you see it acting differently let us know so we can work out why.
User agent * allow is actually redundant as all bots will assume they are allowed unless there is a specific rule for them.
https://github.com/meanpath/robots our robots code is actually open source and in use by a few crawlers.