robots.txt? Yes BUT... Simultaneous with home page grab
(...which was 403'd because my Yahoo-related UAs are case-specific.)
GaryK
10:41 pm on Jun 30, 2009 (gmt 0)
The folks at Yahoo are always attempting some sort of joke when it comes to their UAs and how they crawl. Sadly, their jokes are never funny.
I've given up trying to utilize my contact at Yahoo cause much like msndude he's constantly telling me one thing while corporate goes ahead and does whatever it wants to do.