youfoundjake - 4:20 am on Dec 29, 2007 (gmt 0)
I have 2 sites, one about 2 years old, the other a month. Same exact robots.txt file on both sites. The older site, Slurp obeys. The new site, Slurp plowed right through my site hitting a bot trap that was disallowed in robots.txt, they got banned. Since it's Yahoo, I did unban, but just this once. If they do it again, out they stay!