Forum Moderators: bakedjake
An anonymous tipster wrote to us this morning to tell us that Cuil, the ill-fated “Google Killer,” has unleashed its Twiceler indexing bot on websites across the globe and in the process, has brought many sites down.“I don’t know what spawned it, but when Cuil attempts to index a site, it does so by completely hammering it with traffic,” the tipster wrote. “So much, that it completely brings the site down. We’re 24 hours into this “index” of the site, and I’ve had to restrict traffic to the site down to 2 packets per second, while discarding the rest, or otherwise it makes the site unusable.”
Has anybody here had any problems with Cuil / twiceler hammering their sites into oblivion? If so, what are you doing to counter it? Blocking Cuil?
If it were one tenth true, then why don't the idiots block the bot?
If it were one twentieth true, then maybe, er, another site or two might have reported being 'hammered'.
Chances are, this one anonymous site has a poor server setup, and should be thanking kewell for showing them they have issues.
Even Google gets featured here as 'killing sites by overspidering' - but once you think about it, it's got to be twaddle.
But I do believe the stories that kwoll is ignoring robots.txt and has a pretty cr*p spider ... but that's a matter of record from non-anonymous non-bloggers ;)
