BTWebClient

[koan]

Someone with the user agent BTWebClient/1810 is downloading my whole web site (hundreds of pages) so far by getting some 6-7 pages at once, then pausing 10 minutes or so, and resuming. BTWebClient seems to be some bittorrent application. I'll be adding it to my list of banned user agent.

[keyplyr]

Does your site contain media files or other file types that might attract this kind of interest?

[wilderness]

a google on the UA provides that it's a "torrent bit" downloader.

Does your site (s) offers files that are so large as to require access for such tools?

[koan]

Absolutely not, maybe a small image gallery (20-30 total), but it's mostly textual content, and the bot was going through content pages unrelated to the gallery.

[GaryK]

Hi all. I saw this one last week. It took all the photos from a site I host for one of the admins on my model car site. The site is simple HTML and has no bot-protection mechanisms in place. All in all it took close to 4 GB of photos.