Forum Moderators: open

Message Too Old, No Replies

Yahoo image grabber

         

newbie6

4:46 pm on Apr 2, 2008 (gmt 0)

10+ Year Member



I'm getting far too many hits from
[images.search.yahoo.com...]

User-agents with a robots.txt Disallow: /
Slurp
Slurp/3.0
Yahooseeker
YahooSeeker/M1A1-R2D2
Yahoo-MMCrawler
All my images are in their own directory which has User-agent * disallow.

Is there some new Yahoo bot that might be sneaking by this?

wilderness

8:08 pm on Apr 2, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Most major SE's have such a vareity of tools avialable to users today, that thses SE's literally leave webmasters with limited choices and frustration.

The Yahoo MMCralwer and Seeker have poor records.

However, even Google goes off the deep-end from time to time; I had two crawls from the google image bot last years and this bot has been disallowed in my robots.txt for an eternity.

As an aside; it's been an acceptable practice in this forum to all the use of full IP ranges for the major SE's.

Don