Welcome to WebmasterWorld Guest from 18.104.22.168
Forum Moderators: bakedjake
I've had Problems with the Charlotte-Bot (from searchme.com) - they have taken down my web server repeatedly (admitted, it's a slow machine, but no other search engine has yet managed to bring the machine down).
I've contacted them - no response (after 6 days).
I've added the following to my robots.txt (according to the info on their page):
They fetched robots.txt about every hour, but they ignore their instructions. Therefore, my only choice was to block their network ranges on the IP level (which i have done now) - And i don't really bother about them now, because i have never received a *single* referral from them.
Plus, i do consider their screenshotting technique content theft anyways (contrary to other "visually enhanced" search engines, you can read the full text on their screenshots, so there's no need to visit the original page anymore..
Anybody else experiencing issues with them? My feeling is they are slightly "challenged" in running a crawler... any thoughts appreciated.