dstiles - 6:53 pm on Sep 29, 2011 (gmt 0)

Pfui - my report is from my security logs, generated by the ASP page itself, not from the full site logs, so without more investigation than I have time for at present I do not know if robots.txt was accessed.

Mokita - without being able to read Japanese there is no way of knowing what those UAs are - and they are not, judging from my example, complete.