Harvest/1.8.3

Merged with Harvest 1.9.12

[pendanticist]

64.28.***.** - - [04/Feb/2005:10:58:57 -0800] "GET /robots.txt HTTP/1.0" 200 1774 "-" "Harvest/1.8.3"

[harvest.sourceforge.net...]

[netpreserve.org...] <-NOTE: this is a pdf file.

[newsarchiv.tugraz.at...]

Well, we'll see if it honors robots.txt.

User-agent: Harvest
Disallow: /

Anyone have experience with blocking this one?

[keyplyr]

Harvest-NG web crawler used by search.yahoo.com (also Exalead NG and NG/1.0)

[webharvest.sourceforge.net...]

(now shortened to just 'harvest')

Yes, it comes now and then. No issues so far. But I don't like the name :)

[pendanticist]

If not currently, it was available for download to anyone. Recon that'll be a problem...