GetRight/4.5e

Download manager crawling the net?

[Stefan]

Had this show up for the first time today:

67.18.x.x - - [04/Aug/2004:17:06:57 -0600] "GET /robots.txt HTTP/1.1" 206 1399 "-" "GetRight/4.5e"

A search on GetRight leads to a download manager program of some sort:

Welcome to the home of GetRightŪ
GetRight works with your web browser to help you download files from the Internet.
GetRight can accelerate your downloads and can recover downloads if errors occur (resume downloading). GetRight has many other advanced features such as scheduling, automatic dialing, and file searching.

A site search on WW found nothing.

I'll modify our robots.txt and see if it stays out, but does anyone have any idea what this is about?

[coyote]

Stefan, it's someone using GetRight to download your site.

[Stefan]

Hey thanks, coyote. That occured to me, but I don't understand why it looks for a robots.txt. Maybe it's just being polite?

[coyote]

Hey thanks, coyote. That occured to me, but I don't understand why it looks for a robots.txt. Maybe it's just being polite?

I think some download managers obey robots.txt, or at least look for it before scraping sites.

[TheDoctor]

GetRight is a utility to download user-specified files. You can configure it to override the default routine in IE and NN4, but you can also use it in conjunction with other browsers. It's extremely useful if you're on a dial-up line, since it picks up downloading where it left off if there's an interruption in the connection. It will also close a dial-up connection when it's finished downloading.

Because of these advantages, I use it rather than the download manager that comes with my browsers. It is possible to block it, but I get annoyed if I have to use some other method of downloading a file. Surely what software I run on my PC is my business? Blocking this or that download manager is like blocking this or that browser.

If you don't want files downloaded, don't put them on the web :)

[Stefan]

The reason I denied it in the robots.txt, (which likely won't make any difference), is that bots look for that file, not users. I have enough dodgy bots, scrapers, and thieves sucking back content as it is... I keep my eyes open for stuff like that and make an effort to do something about it, when possible. If it's harmless and just being polite, no problem. I'll undeny it and see what happens.

[TheDoctor]

What file was your user trying to download with GetRight? If it was one that you offer for downloading, or that people try and download with their browsers, what's the problem?

What I didn't say before was that I recommend GetRight on my site to dial-up users (we have a lot of PDF files). I think it's a useful tool, and will remain so until browsers have resumable download managers built-in.

[Stefan]

What file was your user trying to download with GetRight? If it was one that you offer for downloading, or that people try and download with their browsers, what's the problem?

It was a regular htm as I recall. I can't remember, just off-hand, if there were jpgs going out with it.

If it's legit, there isn't a problem. That's why I asked in the first place. I removed it from deny in the robots.txt

Jah guide
Stef

[keyplyr]

I ban all rippers/downloaders.

(just so you have another point of view)

[Leosghost]

HTTrack works in much the same way ..'cept its only for downloading sites ....Banning only works if the things haven't been tweaked to ignore bans

[TheDoctor]

I've not come across HTTrack, but, as Stefan noted, GetRight identifies itself as such and respects robots.txt. Whether you want to ban it or not depends, I suppose, on how ready you are to ignore the needs of visitors on dial-up lines.

Of couse, we wouldn't having this coversation if browsers had decent download manmagers built-in.