iPhone

[keyplyr]

I have a filter that blocks requests from hits that use the same page as referrer, unless certain conditions apply. Likewise if another of my pages are referrer but the IP has not been on the site since the last date change.

However I see a lot of these iPhone hits, from various ranges and a few slight UA variations, all valid looking.

"GET /my-page.html HTTP/1.1" 403 966 "http://www.my-site.com/" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1_2 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) CriOS/39.0.2171.50 Mobile/12B440 Safari/600.1.4"

The question... does iPhone do this typically? Could these not be bots but valid humans? Can't tell by the ranges anymore with cloud services everywhere.

[wilderness]

FWIW, I've iPhone from both Bing and Google denied, and yet they hammer away daily (and for some time), which makes it difficult to view raw logs absent some sort of sorting of the lines (eliminating Bing and Google requests), and I just too lazy to spend the time.

I was getting loads of iPhone requests for single pages, which were also absent supporting file requests. Looked at a lot of them early on and they seemed to be valid requests, thus left them alone (my initial thought was to deny iPhone entirely).

[keyplyr]

My concern is whether the OS or gateway or common app adds a referrer by default. That's why these guys get blocked. Other iPhone users amount to significant traffic/sales for me. So if these guys are legit and only being blocked by something their phone or ISP is doing, I need to know so I can poke a hole.

I was getting loads of iPhone requests for single pages, which were also absent supporting file requests.

That's the rub. Often, when using a mobile app, if the 1st request is a 404 or 403 or anything other than successful, no more requests will be made by the app. So it won't appear like a pageload from a desktop browser with all the associated files that we're accustomed to seeing.

[edited by: keyplyr at 6:36 am (utc) on Jul 27, 2015]

[lucy24]

If your error documents use a stylesheet, you can recognize the human visitors because there's always a request for errorstyles.css. (The favicon is good too, but not all mobiles request it.) Some of them will be infected browsers, but it becomes easy to filter those out.

Some versions of Android Chrome-- I should pin it down but haven't met enough to make it worth the bother-- send the same referer for supporting documents as for the original page. So in logs it looks as if someone went to google search and, by amazing coincidence, landed on every single picture-- and script, and stylesheet-- belonging to the page they requested first.

[keyplyr]

Most apps do not request favicons, and the several iPhone apps I've been watching, will only request the apple-touch-icons if the page loads successfully, and many don't at all. I use simple error response without other connected files.

No issues with Android. Always consistent, easily managed. Just the Apple stuff causing the problems.

[keyplyr]

99.9% probability all these hits (couple hundred a day) are bots with slightly different UA strings, but I had to ask the question. Unless I hear a definitive answer from someone, I'll continue to let them be blocked.

[keyplyr]

I'm seeing much the same with several Android UAs and am now convinced it amounts to spoofing.