jlnaman - 5:49 pm on Apr 22, 2013 (gmt 0)
Lucy24's examples were all for robots.txt. For better or worse, I have been returning a very short (28 bytes) response:
# note the extra <cr> before ';
It is also very useful (to me) for bizarre Bing requests.
The bottom-line is that the number of attempts at non-robot.txt files has decreased.