YaBrowser

[wilderness]

For the most part Yandex has been robots.txt compliant.

They apparently consider YaBrowser a separate bot entity from YandexBot, or perhaps the YaBrowser is beyond the definition of a bot.
Didn't see any robots.text request from the YaBrowser.

199.21.99.218 - - [10/Jul/2015:15:56:25 -0600] "GET /MyFolder/MYPage.html HTTP/1.1" 200 14434 "-" "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/536.5 (KHTML, like Gecko) YaBrowser/1.0.1084.5402 Chrome/19.0.1084.5409 Safari/536.5"
199.21.99.218 - - [10/Jul/2015:15:56:26 -0600] "GET /ImageFolder/FooterImage.gif HTTP/1.1" 200 1938 "-" "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/536.5 (KHTML, like Gecko) YaBrowser/1.0.1084.5402 Chrome/19.0.1084.5409 Safari/536.5"
199.21.99.218 - - [10/Jul/2015:15:56:26 -0600] "GET /ImageFolder/DifferentFooterImage.gif HTTP/1.1" 200 2362 "-" "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/536.5 (KHTML, like Gecko) YaBrowser/1.0.1084.5402 Chrome/19.0.1084.5409 Safari/536.5"

[dstiles]

It's a real browser, made by yandex for general use.

[wilderness]

Many thanks dstiles.
Unfortunately, legit or not it went into my image folders (despite the footer images being inconsequential) which are denied in robots.txt.

Unless they change the IP for their browser their denied.
If they appear with another IP, I'll add the UA.

[lucy24]

made by yandex for general use

Is there some kind of proxying component, causing them to show up from Yandex IPs? wilderness, do they send an X-Forwarded-For header? Most googloid functions (Translate, Preview) do; don't remember about other big-name search engines.

I looked up one site's recent logs, but everything claiming to be "YaBrowser" was either blocked upfront (most often one particular robotic pattern, suggesting preliminary tests) or from a compromised range a/o hinky pseudo-search query.

[keyplyr]

I like it. IMO it is faster and less compromised than Firefox, Chrome, IE or Safari.

That said, there's no such thing as a 'standards compliant user.'

[wilderness]

lucy,
Sorry.
Don't check headers.

BTW, there were two other image requests (same time) from another Yandex IP where I've the Class A denied.
141.8.143.132 - - [10/Jul/2015:15:56:26 -0600]

[dstiles]

199.21.99.218 is within their bot range, I agree, but G and bing both send around bots disguised as (or based on) ordinary browsers in order to get more info on sites. I suggest this is the same thing?

141.8.143.132 - not sure about that range. I have it banned but it could easily be a "broadband" range for their customers; I'm rather heavy-handed on RU and UA ranges. :(

[lucy24]

I'm rather heavy-handed on RU and UA ranges

I set a short-lived cookie (one week, I think) and redirect them to an I'm Awfully Sorry page. If it's a human who really, really, seriously, urgently wants to see the page, they can click on a link and arrive safely. If it's a botnet running off infected browsers, they've placed less of a load on the server than if they'd received the originally requested page.

[keyplyr]

The Yandex Browser has actually been around for a few years now, they just re-released it a couple months ago. I believe the earlier UA was actually: Yandex Browser (then the version number.)

redirect them to an I'm Awfully Sorry page. If it's a human who really, really, seriously, urgently wants to see the page, they can click on a link and arrive safely.

I think that approach may work well on some pages/sites (like yours) but not others, depending on the end user of course.

My personal site is just under 300 pages, almost all being entry pages w/ very good SE ranking and still over 50% of users bounce after the 1st page they land on. Point being, IMO users are impatient. They want what they want immediately. That mouse, or touch pad or finger is just too available.

[lucy24]

The chances are pretty slim that someone from Ukraine* is trying to read an obscure Middle English text, having gotten there using a gibberish search phrase made up of component bits that all happen to occur on the page, but not in any kind of proximity. That slim --but non-zero-- chance is the only reason I even bother with the redirect instead of proceeding directly to a 403.

:: memo to self: check whether RewriteCond uses [OR] (bad range or bogus search) or [AND] (both) ::

* I typed "the Ukraine" and then remembered reading somewhere that Ukrainians absolutely hate this usage.

[keyplyr]

Well our sites are all different of course.

Someone from a Russian forum posted about my site w/ a link and it proliferated, so for the last year I get 30 to 50 UA/RU humans a day. These are mostly legit users. Since I publish Adsense and M$ ads, these guys have value.

Point is; UA/RU users make all the URL typos and other mistakes that anyone else would and I no longer consider country of origin a factor in determining the validity of a request, just as long as they're human... and of course if they're female, they have to be hot!