curl/7.7.1

no robots. grabbed my odp dir

[bull]

66.181.168.66 - - [11/Nov/2002:10:45:04 +0100] "GET /odp/ HTTP/1.1" 200 2847 www.me.net "-" "curl/7.7.1 (i386--freebsd4.3) libcurl 7.7.1 (SSL 0.9.6) (ipv6 enabled)" "-"

resolves to likewhoa.net. for my taste too often the word "adult" is used on their main page... i blocked it.

[seindal]

Curl is a general purpose download program, much like wget. It doesn't ask for robots.txt because it is not a robot.

René.

See: [curl.sourceforge.net...]

[bull]

in this case it _acted_ like a bot: no referer, getting odp dir. perhaps a modified version? but likely not a human user.

[seindal]

Many programs are written based on the curl sources, so it could be anything. Both referrer and user-agent are easily faked with these programs, but you could block the IP or try with the user-agent.

There is a very long thread on user-agent blocking on
[webmasterworld.com...]

René

[Finder]

bull,

It hit my site too and was definitely a bot since it couldn't handle a URL with an encoded ampersand. I didn't like their site either so I don't really care what they are up too, I'm blocking them.

[bull]

66.181.168.66 - - [13/Nov/2002:21:17:55 +0100] "GET /odp/ HTTP/1.1" 403 2126 www.me.net "-" "Mozilla/8.0" "-"

followed the 2 links on that page but not any further (with 2 sec delay). --- oh this is likewhoa.net again. sure this is not a bot, seindal? ;-)

[fiestagirl]

I've also got them visiting, only this time they say that it's a robot:

66.181.167.13
Demo Bot DOT 16b

Looks like Like Whoa Inc owns 66.181.160.0 - 66.181.191.255