CreativeCommons

Forum Moderators: open

Message Too Old, No Replies

CreativeCommons

wilderness

5:34 am on Nov 26, 2004 (gmt 0)

67.15.80.14 - - [25/Nov/2004:16:07:42 -0800] "GET /robots.txt HTTP/1.0" 200 2835 "-" "CreativeCommons/0.06-dev (Nutch; [nutch.org...] nutch-agent@lists.sourceforge.net)"

This IP range and provider was discussed in another thread.
I poked around in the archive with a few searches with no success.

My records have the bot previously crawling unidentified.

volatilegx

2:16 am on Nov 28, 2004 (gmt 0)

> My records have the bot previously crawling unidentified.

You mean same IP, no User Agent?

wilderness

3:50 am on Nov 28, 2004 (gmt 0)

This provider is a haven for unidentified crawls:

66.98.128.33 - - [11/Jun/2004:05:21:18 -0700] "GET /robots.txt HTTP/1.1" 206 2365 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"

67.15.4.9 - - [03/Apr/2004:05:28:19 -0800] "GET /myfolder/mypage.html HTTP/1.1" 200
42747 "-" "rbvyljilkfvcelahcjgnn gyabgcsumaimk"

67.15.50.18 - - [25/May/2004:12:58:56 -0700] "GET /myFormerbot trap
HTTP/1.0" 404 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET
CLR 1.1.4322; .NET CLR 1.0.3705)"

64.246.0.17 - - [21/Jun/2002:18:18:28 -0700] "GET / HTTP/1.1" 200 14146 "-" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT)"

ranges are just as irrelavant to me as they are to this provider and its customers.