Crazywebcrawler

[keyplyr]

UA: CRAZYWEBCRAWLER 0.9.10, http://www.crazywebcrawler.com
Protocol: HTTP/1.1
Host: Various (distributed)
Robots.txt: No

CrazyWebCrawler-Spider is the user-agent used by CrazyWebCrawler, a web
crawling service provider. It allows its users to design and run custom web crawls.
So, if Spider is crawling your website, it means that one or more of our users created
a web crawl that went (eventually) to your website.

People use CrazyWebCrawler for a variety of reasons, including providing data to
their own search engines, monitoring trends in online opinions, and other
interesting applications.

"Interesting applications" - I bet.

[edited by: Ocean10000 at 1:50 pm (utc) on Oct 2, 2015]
[edit reason] broke unintenial link [/edit]

[lucy24]

9.10 eh? I find them ranging from 0.9.0 (October of last year) through 0.9.9 (recent), suggesting a weird reluctance to emerge from beta and admit to the full 1.0.0. Version numbers are sequential-- that is, later date = higher number-- which is a tiny bit worrying because it implies they've got some central, uh, whatever it is that robots have. Intelligence? Nerve stem? Evil overlord?

[keyplyr]

RE:
[edited by: Ocean10000 at 1:50 pm (utc) on Oct 2, 2015] [edit reason] broke unintenial link [/edit]

Sorry... the url in the UA did not display as active in the editor preview or I would have deactivated it. Then of course I never got to see how the post actually displayed since new posts in this forum are hidden until approved. Catch 22 I'd say.