Mokita - 11:37 pm on Dec 28, 2007 (gmt 0)

volatilegx wrote:

What do you mean? You can't require them to ask for files via robots.txt.

Sorry, I see now that my sentence is somewhat ambiguous.

What I meant to say, is that normally Yahoo Slurp correctly and properly obeys our robots.txt, which disallows all images and style sheets.

On the one occasion shown in the logs posted above, Slurp requested all supporting files, which it should not have if respecting robots.txt - as Yahoo claims it does.