Forum Moderators: open

Message Too Old, No Replies

Cuil in stealth mode?

         

keyplyr

8:31 am on Sep 18, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



UA: blank
Referrer: blank
robots.txt: yes
IP address: 216.129.119.**
rDNS: ramp2b.cuil.com
[Verified]

Requested 15 web pages, all 403 because of sneakiness. This is the first time I've seen this behavior from Cuil.

Pfui

1:10 pm on Sep 18, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



1.) Wow, only four Google references for "ramp2b.cuil.com". Must be pretty darned new -- and hopefully not a taste of things to come.

2.) The majority of my hits come from the likes of the gazillion-results "crawl-16c.cuil.com" (67.218.116.nnn) and are properly behaved.

HOWEVER...

3.) Two days ago, I spotted the following at least twice -- perhaps not so coincidentally, it's also from Layer42.net, as are all of the IPs in this thread, ditto .cuil.com itself:

67.218.99.nnn
robots.txt? YES

(That was blocked from the get-go because of the blank UA.)

Hmm. Everything's cuil- and Layer42-related. Doesn't bode well, huh?

jdMorgan

1:56 pm on Sep 18, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Cuill's Twiceler continues to fail to recognize multiple-user-agent policy records in robots.txt, indicating a lack of attention to detail at the most basic level of crawler implementation. So it wouldn't surprise me if they simply forgot to populate the User-Agent header...

Jim

keyplyr

10:05 am on Sep 19, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Hmmm, came again 24 hours later (to the minute) same behavior, same 15 attempts.