Yahoo-Test ignoring robots.txt

Forum Moderators: open

Message Too Old, No Replies

11:58 pm on Dec 10, 2008 (gmt 0)

Just got this hit to a disallowed page...

67.195.37.*

Mozilla/5.0 (Yahoo-Test/4.0; mailto:vertical-crawl-support@yahoo-inc.com)

Now blocked by other means.

2:22 am on Dec 11, 2008 (gmt 0)

We had a whole bunch ignoring robots file from 68.180.201.* at the end of November.

6:27 am on Dec 18, 2008 (gmt 0)

I sometimes see hits checking webmail links. Is is possible someone put a link to that specific page in an email?