htdig

[wilderness]

www.htdig.org/

I've had them denied for a week or so.
They are persistent
204.144.252.1

(edited by: engine at 9:52 am (utc) on April 11, 2002)

[Josk]

Do you mean htdig.org, or 204.144.252.1. For me htdig.org resolves to 216.136.171.204 and is a seperate netblock. You're not getting confused by someone running a htdig spider>

[wilderness]

Hey Josk,
I'm not confused ;-) Except when I try to find my way ;-)
www.htdig.org/ is a software spider.
I don't recall reading that htdig had its own bot?
Similar to ia_archiver and the others where the ip routes directly to the software's IP.

The 204 IP routes to soemthing named Starsite which has a Colorado backbone provider.

[littleman]

Htdig is used by many reputable institutions, universities and such. Be careful about banning via user agent.

[wilderness]

<snips>Be careful about banning via user agent.>

Many thanks littleman.
This particular user from a Colorado IP.
Colorado seems to have a fascination with
horses even if its a breed their not accustomed to ;-)

[PoolDoc]

An rDNS search suggests ViWire.com (AKA searchlogic.com) seems to own the entire 204.144.252.1/24 Class C, or "204.144.252." in Apache-speak.

Checking their website,
[searchlogic.com...] ,
suggests they are trying to run a brand snoop bot, and probably (IMHO) deserve banning.

The fact that they are using htDig to do it, suggests they are likely to have a short and unprofitable life on the 'Net, since htDig is a moderate performance open source search tool that doesn't begin to be up to the task of searching the Net world wide.

But, there's almost no chance that the good folks at [htdig.org...] have anything whatsoever to do with ViWire.