jdMorgan - 5:16 pm on Jun 12, 2006 (gmt 0)

> Did you try the Yahoo! Slurp China variant? I wonder if that makes any difference?

No, I'm not going to experiment with a live site beyond what I've done -- Yahoo said it should work with "Slurp" and/or "Slurp China" and that's been disproven in my test. The ball is in their court, unless they want to hire me as a consultant <joke>.

They need a serious re-write of their robots.txt page to fully explain the various crawler variants, their functions, their robots.txt names, and the interactions between robots.txt records when using multiple Slurp control records. This page should be as universal as possible, and as noted by IncrediBill and myself above, should be available in multiple languages.

Jim