"pavuk/0.9pl27 i686-pc-linux-gnu"

What is this?

[Ulrike]

I was hit very hard by this thing. I think it took every page from my site including the messageboard. My logfile from the 6th is three times as big as usually, common spider actitivity included. My webhost disabled my messageboard script.

128.130.175.82 - - [06/Aug/2001:17:13:59 +0000]
"GET /UB/UltraBoard.cgi?action=Search&SID=8680 HTTP/1.1" 200 10964
"http://www.dragonlibrary.com/UB/UltraBoard.cgi"
"pavuk/0.9pl27 i686-pc-linux-gnu"

What is that? A spider or another program? Does it follow robots.txt or must I ip-ban it?

[sugarkane]

Pavuk is a site mirroring utility for *nix. It looks like someone liked your site enough to download the whole thing to browse offline...

AFAIK it doesn't respect robots.txt.