searchme.com ignoring my robots.txt

Problems with Charlotte bot ignoring robots.txt contents

[nonanet]

Hello,

I've had Problems with the Charlotte-Bot (from searchme.com) - they have taken down my web server repeatedly (admitted, it's a slow machine, but no other search engine has yet managed to bring the machine down).

I've contacted them - no response (after 6 days).

I've added the following to my robots.txt (according to the info on their page):

User-Agent: Charlotte
Disallow: /
Crawl-Delay: 30

They fetched robots.txt about every hour, but they ignore their instructions. Therefore, my only choice was to block their network ranges on the IP level (which i have done now) - And i don't really bother about them now, because i have never received a *single* referral from them.

Plus, i do consider their screenshotting technique content theft anyways (contrary to other "visually enhanced" search engines, you can read the full text on their screenshots, so there's no need to visit the original page anymore..

Anybody else experiencing issues with them? My feeling is they are slightly "challenged" in running a crawler... any thoughts appreciated.

[tangor]

Gave them a try. Liked it when one of the fellows came on the list and chatted... but after so many months of letting them beat me up bandwidth-wise and NOT ONE REFERRAL just gave 'em the boot via robots.txt. Will wait and see if that is honored.

[Rosalind]

I've also banned them via robots.txt on some sites, and I'm waiting to see if the situation changes in any way. My guess is they will not take off, because of the screenshot infringement. It's not as though sensible webmasters will be falling over each other to recommend them to other people.