This picsearch bot is ultra dumb, I can't imagine why MSN works with them.
psbot/0.1 (+http://www.picsearch.com/bot.html)"
This bot hits 20k+ 403 forbidden at my site and continues to do so. What a dumba**.
Mokita
11:34 pm on Nov 11, 2006 (gmt 0)
Have you tried giving it access to robots.txt and disallowing the psbot? Works well for me. I have it disallowed in all our sites, and I rarely ever see it now.
User-agent: psbot Disallow: /
Possibly because it isn't finding robots.txt, it thinks it's OK to crawl and index your site, so keeps on trying.
SEOPTI
6:47 pm on Nov 12, 2006 (gmt 0)
Thanks for the info. I need to start using robots.txt, I ban them first because most of them do not respect robots.txt
The next poor coded pest in my logs I see is: CazoodleBot/Nutch-0.9-dev (CazoodleBot Crawler)