Forum Moderators: open

Message Too Old, No Replies

Helix/1.2

         

bull

7:53 am on Aug 19, 2005 (gmt 0)

10+ Year Member



65.110.21.171 - - [19/Aug/2005:09:08:25 +0200] "GET /robots.txt HTTP/1.1" 200 4604 "-" "Helix/1.2 (+http://www.sitesearch.ca/helix/)" "-"

ODP-based crawler. Seems to obey robots.txt

wilderness

4:24 pm on Aug 19, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



appears to be anoter DMOZ clone.
I didn't care for the references on their website and expaned a range (up to the backbone) that I previously had denied.

65.110.21.171 - - [19/Aug/2005:02:57:04 -0700] "GET /robots.txt HTTP/1.1"
200 3473 "-" "Helix/1.2 (+http://www.sitesearch.ca/helix/)"

GaryK

12:16 pm on Aug 21, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Helix visited my sites this week too.

The same crawler with a different user agent but the same IP Address visited my sites this past week:

Mozilla/2.0 (compatible; DC9FE0029G; FreeBSD 5.4-RELEASE; i386; en_US)
65.110.21.171

Both of them read robots.txt twice and then left.

They all trace back to a project run by someone named Marty Anstey. According to his website this Helix project used to be known as Reaper, another ua I've had banned for a long time.