Forum Moderators: open

Message Too Old, No Replies

BTWebClient

         

koan

12:23 am on Nov 22, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Someone with the user agent BTWebClient/1810 is downloading my whole web site (hundreds of pages) so far by getting some 6-7 pages at once, then pausing 10 minutes or so, and resuming. BTWebClient seems to be some bittorrent application. I'll be adding it to my list of banned user agent.

keyplyr

9:11 am on Nov 22, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Does your site contain media files or other file types that might attract this kind of interest?

wilderness

2:17 pm on Nov 22, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



a google on the UA provides that it's a "torrent bit" downloader.

Does your site (s) offers files that are so large as to require access for such tools?

koan

11:42 am on Nov 23, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Absolutely not, maybe a small image gallery (20-30 total), but it's mostly textual content, and the bot was going through content pages unrelated to the gallery.

GaryK

5:00 pm on Nov 23, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hi all. I saw this one last week. It took all the photos from a site I host for one of the admins on my model car site. The site is simple HTML and has no bot-protection mechanisms in place. All in all it took close to 4 GB of photos.