larbin ([e-mail])

Anyone seen this before

[fathom]

66.28.68.235 - - [12/Jul/2002:04:00:16 -0400] "GET /robots.txt HTTP/1.0" 200 166 "-" "larbin ([e-mail])"

[mack]

Larbin is an indexer available under GNU

You can find out more here
[larbin.sourceforge.net...]

[fathom]

Thanks Mack, I'll be excluding it.

[mack]

Not sure if you need to exclude it.. (unless it's been misbehaving) could be a new se in the brewing.

who knows :)

[jdMorgan]

On the other hand, we've had dozens of hits from larbin variants, and none have
been well-behaved or have shown any sign of a "human" access pattern. Only one
included contact information in the UA, and it was bogus.

Same goes for Indy Library and several others.

I have banned all UAs with "larbin" occurring anywhere in the UA string, and
after thoroughly reviewing my logs, have not seen anything that would make me
change my mind.

<rant>
My site is for human visitors. If automated agents want to use our bandwidth,
they need to demonstrate some immediate benefit to my users or to my
organization - For example, established search engines are free to index us at
will. But not e-mail spammers, site downloaders, or other bandwidth leeches.

I used to be more open-minded, but one month of paying excessive-bandwidth
penalty changed my mind...

Standard browsers are welcome. SE robots are welcome. Anybody else needs
valid contact info (preferably a web page) in their UA, or sorry - 403.

If I had "pull" in Washington, D.C., I'd ask for a law making forging a UA
or failing to provide valid contact info a crime, and require some
standardization in User-agent formats - at least in the U.S. Eventually,
international law is going to have to comprehend these problems - Now, it
is mostly a nuisance, but as it grows, it will become clearer that bandwidth
leeching is theft. I know - I had to pay the bill, and we're a non-profit.
</rant>

Sorry... it's late!
Jim