iFile

iPhone file downloader & file manager

[keyplyr]

Being a die-hard Android user, I'm not too familiar with some of the iPhone specific apps. Just caught this thing scraping my pages. The user connected to my server with a normal iPhone UA:

Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_1 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Mobile/11D201

Then the user found something they wanted to scrape and save to their phone (or send to their computer) this UA connected:

iFile Browser 7.1 rv:81 (iPhone; iPhone OS 7.1.1; en_PH)

Did HEAD checks, then GET requests scraping html, images, css, scripts, et al.

From the app page:

iFile is an app designed to allow you to move files from computers (even between different operating systems) and your iPhone or iPad. Requires iOS 7.0 or later. Compatible with iPhone, iPad, and iPod touch. This app is optimized for iPhone 5, iPhone 6, and iPhone 6 Plus.

[Samizdata]

I have used iFile, it is a file manager for iOS which has recently added the ability to download files to an iPhone or iPad from the web - a function that is not built-in by Apple.

Such apps (there are better ones) largely serve as a substitute for the standard computer browser function of right-clicking a link and choosing "Save As" or similar.

The idea that they scrape websites is nonsense, unless Internet Explorer, Firefox, Chrome, Opera, Safari and other browsers are put in the same category.

But webmasters are entitled to block anything they want.

...

[keyplyr]

The idea that they scrape websites is nonsense,

I just said it scraped pages of mine.

The tool is not a fault. Like most things, it is the user.

[Samizdata]

I just said it scraped pages of mine

I would say a person saved pages from your website, as they might do on a desktop or laptop computer.

I wouldn't call it scraping, every major browser has offered the function for years (and they have a Print option too).

I generally use what are known as "download apps" on the iPhone to save media files (PDF, MP3, MP4 etc) that webmasters have specifically made available for that purpose - there is no native iOS method of doing it, even though downloadable files are a staple feature of websites.

That is clearly not scraping either.

...

[keyplyr]

Saving html, images, scripts and css files is SCRAPING when my copyright notice is clearly displayed. Some users, like the guy who hit my site, will abuse this tool.

I 'm not singling out iPhone at all. There are hundreds of file scraping tools out there. I just found a new one and have blocked it across the sites I manage. My sites, my rules.

What is alarming about this one is the ability to send these files to remote computers.

[Samizdata]

There are hundreds of file scraping tools out there

What they have in common is that they are automated.

This tool, as explained, is the equivalent of the "Save As" function used by humans in all regular desktop browsers.

Perhaps you should block those too.

What is alarming about this one is the ability to send these files to remote computers.

There is nothing alarming about networking your devices.

When I got an iPhone, finding out how to download files from the web, send them to the built-in apps, and copy them to my desktop and laptop computers was almost the first thing I did.

(The actual first thing I did was find out how to use it as an effects unit for my guitar, but that's another story).

My sites, my rules.

I would never argue with that.

But your definition of a "scraper" is idiosyncratic, to say the least.

Don't forget to block IE, Firefox, Chrome, Opera and Safari - they all seem to meet it.

...

[keyplyr]

Don't quite understand why so are being so defensive. I just reported that I found a download tool, I had previosly been unaware of, scraping my site. I never said it was automated. A scrape is a scrape.

Yes I do block other tools like this. They are all unwelcome on my personal site where I want the user to buy my books, not scrape it for free. IMO this is blatant theft since it is very clear that the content is restricted and must be purchased.

On the other sites I manage, my clients have made been explicit about unauthorized fiile downloading.

Your understanding of the term "scraper" seems quite different than mine and the basis for our disagreement. However one purpose of this forum is to identify User Agents. I have done that. The back story of how I discovered this UA is of couse my interpetation of those events.

[Samizdata]

one purpose of this forum is to identify User Agents

Another is to explain what the associated tool does.

iFile and other apps like it allow iOS users to save pages and files in the same way that every desktop browser does - and nobody calls those "scrapers".

Apple provides no built-in mechanism on iOS for downloading files from the web, and only provides for file transfers between devices using iTunes or iCloud.

But they also famously say "There's an app for that", and there are many third-party applications that add traditional computer functionality such as the time-honoured method of downloading publicly available files from websites and the simple wi-fi networking that you find so "alarming".

Your understanding of the term "scraper" seems quite different than mine

It certainly is, and I am very happy about it.

Saving html, images, scripts and css files is SCRAPING

Every desktop browser does it, and has done for years.

I don't block them, you are free to do so if you want.

Don't quite understand why so are being so defensive

I am not defending anything, I am explaining basics.

For the benefit of others, obviously.

...

[keyplyr]

As one of the NOC supervisors at our university, I've also decided to add this UA to the bock list across the network. The potential for misuse is too great.

[Samizdata]

The potential for misuse is too great.

This tool merely allows iOS users to save webpages and download files (PDF, DOC, XLS and other university favourites) in the same way that all existing desktop browsers have done since the mid-1990s.

To be consistent you should block all web browsers.

Saving html, images, scripts and css files is SCRAPING

It may come as a surprise that all web browsers work by downloading and saving html, images, scripts and css files to the viewer's hard drive - nobody can look at your webpages otherwise.

Scrapers are a serious problem for many webmasters, but by effectively defining the term "scraper" as "anything that can view a website" you are trivialising the issue and making yourself look rather silly.

...

[keyplyr]

making yourself look rather silly.

No need to get personal Samizdata. Let's keep things on a professional level here. Thanks.