Welcome to WebmasterWorld Guest from 54.196.244.186

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

Cuil

     
11:26 am on Jul 28, 2008 (gmt 0)

Junior Member

10+ Year Member

joined:Oct 23, 2003
posts: 103
votes: 0


Anyone know the Agent for the new search engine Cuil?

[cuil.com...]

6:18 pm on July 28, 2008 (gmt 0)

Administrator from US 

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 25, 2005
posts:14624
votes: 88


It's "twiceler" and they have more information about it here:
[cuil.com...]

You can also find lots of posts with details about it by searching WebmasterWorld [google.com].

6:56 pm on July 29, 2008 (gmt 0)

Administrator from US 

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 25, 2005
posts:14624
votes: 88


Just in case anyone missed it, Cuil launched [webmasterworld.com] to mostly unfavorable reviews.
7:11 pm on July 29, 2008 (gmt 0)

Junior Member

10+ Year Member

joined:Oct 23, 2003
posts:103
votes: 0


Thanks Bill, that's why it's worrying me. It sucks bandwitdh and it spiders the most crazy things.

I am running a big forum, and it only spiders irrelevant things like "post" "print" and things like no other spiders do (blocked by roboats.txt and the software itself).

Will block them now I think.
Cheers

5:08 am on Sept 15, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 17, 2002
posts: 2251
votes: 0



System: The following message was spliced on to this thread from: http://www.webmasterworld.com/search_engine_spiders/3744847.htm [webmasterworld.com] by incredibill - 1:37 pm on Sep. 16, 2008 (PST -8)


Twiceler/Cuil used to be hosted at Hurricane Electric. It's not anymore. Do you all still consider it a bot worthy of being banned? At least this time when it visited it read robots.txt.

BTW, I laughed myself silly at this claim on their home page:

Cuil is the biggest search engine on the planet.

10:46 pm on Sept 16, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
posts:25430
votes: 0


They simply claim to have indexed more URLs than anyone else, so the claim is as valid as any advertisement you might read, see, or hear...

Only time will tell, but at least they've now published their index, which makes it easier to justify allowing them to crawl our sites.

Jim

11:36 pm on Sept 16, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 17, 2002
posts: 2251
votes: 0


Jim, it's one thing to claim to have indexed more URLs than anyone else and being the biggest search engine on the planet. :)

blocked by roboats.txt

Is this a special file for maritime industry websites? LOL ;)

~gary.

11:55 pm on Sept 16, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member Top Contributors Of The Month

joined:Aug 29, 2006
posts:1312
votes: 0


Is this a special file for maritime industry websites?

It might be Cuill's cunning response to Google's patented floating datacenters.

...

9:59 am on Oct 15, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 27, 2003
posts: 1642
votes: 0



System: The following message was spliced on to this thread from: http://www.webmasterworld.com/search_engine_spiders/3765969.htm [webmasterworld.com] by incredibill - 12:53 pm on Oct. 15, 2008 (PST -8)


Had to ban the cuil agent Twiceler last night when it was happily trawling through directories that everyone else seems to manage to stay out of, as instructed by the robots.txt

Anyone else noticed this?

Bad Cuil, bad! :(