Frank_Rizzo - 5:28 pm on Jun 24, 2010 (gmt 0)
Are we sure this is a bot? Even if it is a bot why does it wish to know if sensitive pages do not, or do exist on a site?
My 'sensitive' pages are secured us much as I can make them. It still doesn't make it right for Yahoo (or a rogue 3rd party) to go tapping the walls to find out where the safe is.
It is sniffing around and I don't like it. It should crawl only the pages it knows about and nothing else.