Onavo Proxy

Mobile data compressor

[dstiles]

By no means new but I've just come across this. It compresses mobile data via a server-based proxy. The first article I found states it uses Amazon cloud as the proxy server but all I've got in my logs over the past 6 months is Internap.

No problem allowing this through but I have to make exceptsions for the servers. Does anyone know if Onavo uses any service other than Internap?

[keyplyr]

Domain record says they use Amazon:
54.230.0.0 - 54.231.255.255
54.230.0.0/15

From onavo.com it appears the proxy service is offered through a couple apps and not from the ISP mobile network. That's the good news.

The bad news is it's caching my content and selling to the user via a "compression service" mobile app. This is something I have always been against no matter what they call it. IMO this, and all caching of my property, is copyright infringement. This company aims to make a profit from my content without my permission nor any payment to me. They stay blocked.

Please post any/all ranges when discovered.

[trintragula]

This kind of compression service is built into some mobile phone internet services for bandwidth reduction. The customer gets it by default.

[keyplyr]

@ trintragula - Please supply sources. Saying "some mobile phone internet services" doesn't really help anyone.

[trintragula]

Google "phone image compression"

O2, vodaphone, T-mobile all do this. Probably others do. It may be 3G only, but I don't know.

Note some people will be using alternative proxies designed to *avoid* this and just send the uncompressed image.

If you want a specific link there's this:

[stackoverflow.com...]
or this:
[nokiausers.net...]

I think the reason is that most websites don't resize images for mobile devices, so they do this to make people's phone-browsing experience tolerable in those cases. Otherwise people would be waiting minutes for pictures to arrive sometimes.

My mobile phone is still in the stone age, and doesn't do web browsing beyond WAP, so I've only really experienced this from the server end (except when I borrow my daughter's phone for testing)... :)

[keyplyr]

I see no evidence T-Mo caches content and serves it from a proxy. I haven't tested 02 or Vodaphone.

[trintragula]

Hmm, I'm not seeing any evidence that Onavo does either...

It looks like it's just a pass-through compression proxy, run as a separate service rather than through the mobile phone operator.

The web site talks about data saving, but I'm assuming they're talking about reducing your data usage bill on your phone, not stashing web content.

Maybe I'm missing something...

[dstiles]

keyplr - yes, probably the same commentary I found. :)

However, as I said, no sign of amazon in the past 6 months, only internap. Presumably they've switched clouds (the original rained out?!) but I was wondering if there were others out there. Thanks, anyway.

[keyplyr]

@trintragula Data saving means they save your data, simple as that. In order for them to compress your files, they need to have your files. This is not new, just the same old pig wearing a different hat. Caching is stealing IMO.

[trintragula]

I think in this context "data saving" just means they're saving (conserving) your data allowance on your phone service. I don't think it means they're caching. At least I didn't get that impression from their website.

[keyplyr]

I disagree. They even tell you they cache our files.

From onavo.com > Inavo Extend for Android:

data caching means you won't have to use your data to load the same images every time

Because it is now cached on their server.

According to SpamBotSecurity.com, when onavo scrapes it will first make a head request for each image file using this UA "ICAP-IOD". Since there are at least 4 (possibly 5) different apps in play, there may be other UAs.

IP Blacklist reports that ICAP-IOD does not obey robots.txt (duh)

Hostname: secure.onavo.com
User Aget: ICAP-IOD Bot
Ignores robots.txt
Crawling for images – no other content

FYI, onavo is now owned by Facebook.

[trintragula]

It looks like they're caching on the phone, not on their servers. Which is what a browser does anyway.

Revolutionary universal data caching means you won't have to use your data to load the same images every time you visit a website or use an app because they'll already be available to you on your phone. And don't worry about local storage, because you control the percent of your phone's memory used for storage.

I italiced the bit you quoted, and bolded the bit about storing on the phone.
When they say "use your data" they're talking about spending money for bandwidth.

Universal in this context is I think bounded by the phone. It probably just means the cache on the phone is available to all apps and browsers.

I really don't think this is anything to worry about. So long as they follow the standard HTTP caching rules (and they'll run into very obvious usability problems if they don't) it'll be no different than using a standard browser.

[trintragula]

Following up on your post edit:
A service like this would not regard itself as a robot, and would presumably only be requesting images that a phone would have just requested. It would thus not have cause to look at robots.txt, any more than a browser would.

If they're actually running an independent bot for some other purpose, that's a different story...

Looking at my logs, I have actually seen a handful of image requests from this thing in the last 6 months. Certainly nothing approaching scraping.
All internap (voxel dot net).
Ranges: 216.223.27/24, 209.191.164/24 and 31.186.228/24
No AWS.
The UA I've seen is:

ICAP-IOD

No mention of 'bot' in the UA.

[keyplyr]

Believe what you want. There's a lot of documentation in forums & blogs about them. They crawl your site, scrape your images (cache) then compress them and use it as their product through their mobile app. Whether the images are then stored on the users phone or not is of no importance.

This is a huge difference from what a browser does. This is a company modeling their business with my property without my permission. They stay blocked on all sites I manage. End of story.

BTW - Those internap ranges are actually:
31.186.228.0/23
209.191.164.0/23
216.223.24.0/22

[dstiles]

Are you certain about the sub-ranges, keyplr? There is nothing in DNS to show it.

I opened up all internap ranges I have to allow the proxy (and nothing else) at any IP,

[keyplyr]

@dstiles - Sorry, did not intend to imply these ranges were restricted to proxy, only that these were the more accurate internap ranges of those that were posted.