Forum Moderators: open
User-agents with a robots.txt Disallow: /
Slurp
Slurp/3.0
Yahooseeker
YahooSeeker/M1A1-R2D2
Yahoo-MMCrawler
All my images are in their own directory which has User-agent * disallow.
Is there some new Yahoo bot that might be sneaking by this?
The Yahoo MMCralwer and Seeker have poor records.
However, even Google goes off the deep-end from time to time; I had two crawls from the google image bot last years and this bot has been disallowed in my robots.txt for an eternity.
As an aside; it's been an acceptable practice in this forum to all the use of full IP ranges for the major SE's.
Don