homepage Welcome to WebmasterWorld Guest from 54.198.42.213
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld
Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
YaBrowser
lucy24




msg:4628527
 2:18 am on Dec 7, 2013 (gmt 0)

Anyone have any idea what the YaBrowser is? I've seen it infrequently over the past year, often but not always attached to Ukrainian robots. But today's logs caught my eye with a series of requests beginning like this:

93.158.151.25 - - <date> "GET <page> HTTP/1.1" 200 8415 "-" "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/536.5 (KHTML, like Gecko) YaBrowser/1.0.1084.5402 Chrome/19.0.1084.5409 Safari/536.5"

and continuing through a few more pages. Seemingly humanoid BUT

-- IP varied randomly between 93.158.15[01].2x and 178.154.243.1xx *
-- no referer for image requests, but otherwise appropriate for page
-- <noscript> version of each page's piwik file
-- no favicon (or apple-touch-icon)
-- NO STYLESHEETS (should have been a total of four)

Everyone recognize those IP ranges? They're both Yandex; I've met them many times before.

I've withheld one possibly significant piece of information.


* ^93\.158\.15(0\.2[01]|1\.2[45])\b and ^178\.154\.243\.1(0[4-9]|10)

 

keyplyr




msg:4628711
 8:07 am on Dec 8, 2013 (gmt 0)

Yandex.Browser is a freeware web browser that uses the WebKit layout engine and is based on the Chromium project. The browser checks webpage security with the Yandex security system and checks downloaded files with Kaspersky anti-virus. The browser also uses Opera Software's Turbo technology to speed web browsing on slow connections.

These variations are reported:

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/536.5 (KHTML, like Gecko) YaBrowser/1.0.1084.5402 Chrome/19.0.1084.5402 Safari/536.5

Mozilla/5.0 (Windows NT 5.1) AppleWebKit/536.5 (KHTML, like Gecko) YaBrowser/1.0.1084.5402 Chrome/19.0.1084.5402 Safari/536.5

Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/536.5 (KHTML, like Gecko) YaBrowser/1.0.1084.5402 Chrome/19.0.1084.5402 Safari/536.5

lucy24




msg:4628716
 9:38 am on Dec 8, 2013 (gmt 0)

Well, that last one certainly sounds familiar ;) In fact I did wonder if they were a sort of Russian Chrome. Any ideas what they were doing? Picking up images but not stylesheets is distinctly bizarre. There weren't any concurrent requests for the same files from elsewhere, the way you'll often see with an AV type of thing.

Angonasec




msg:4628728
 12:43 pm on Dec 8, 2013 (gmt 0)

We've simply blocked all 93. for years, together with the US ranges they are migrating to.

Leaving the door open is asking for headaches.

keyplyr




msg:4628768
 6:12 pm on Dec 8, 2013 (gmt 0)



Lucy, AFAIK YaBrowser is just a web browser built on the Chrome engine that is offered by Yandex. Nothing organically malicious about it. It is offered for DL at several web sites.

Now, that being said, Yandex *may* also being using this UA for something we don't yet understand, much like the new Google UAs that have been popping up lately.

As for blocking Yandex ranges, I have seen noting that has alarmed me about Yandex. I have always allowed all their ranges and get a little traffic from both their Russian and American branches.

I *do* see increasing mischief from Russian Telecom and Mobile Carrier ranges, but I don't associate that with Yandex.

wilderness




msg:4628772
 6:41 pm on Dec 8, 2013 (gmt 0)

We've simply blocked all 93. for years


Ditto (i. e., "Might Mean Marie" in this same forum from ages ago)

blend27




msg:4628775
 7:03 pm on Dec 8, 2013 (gmt 0)

We've simply blocked all 93. for years


Close to 15K in sales in the past 3 month from 93. ranges: UK, FR, IT, DE, FI, AT, IE, PT. $30-$80 widgets, no it's not goat cheese samples.. :)

Do RDNS on ranges within, see who the ISPs are.

I love me some RU and UA ranges blocked, but the entire 93.?

lucy24




msg:4628816
 10:59 pm on Dec 8, 2013 (gmt 0)

Money isn't an issue for me. Because, ahem, there isn't any. I do tend to follow a one-strike policy on eastern Europe:* if I meet a robot from anywhere in the area, I don't really care whether the range is nominally humans or servers. But in this particular 93.subsector I already know it's Yandex so I will wait and see.


* Do not ask me to explain why "eastern Europe" includes Poland but excludes Hungary and the Czech republic.

Angonasec




msg:4628901
 5:10 am on Dec 9, 2013 (gmt 0)

"... but the entire 93.?"

Well, if you're chasing pennies, don't let me stop you, but as you point out, blocking 93. reinforces the barricade against EU detritus too: A definite plus.

Angonasec




msg:4628902
 5:16 am on Dec 9, 2013 (gmt 0)

We've blocked Yandex since they began, indeed, before the establishment of the Ukrainian Hackers University.

You, however, may want your site listed in Yandex.

wilderness




msg:4628907
 5:28 am on Dec 9, 2013 (gmt 0)

We've blocked Yandex since they began


It's not really necessary, as Yandex is robots.txt compliant

Angonasec




msg:4628931
 9:18 am on Dec 9, 2013 (gmt 0)

Yandex failed to comply with our robots.txt, and they now attempt to take screen-shots from US cidrs.

Factors which mitigate not the chopper falling on their fat necks.

dstiles




msg:4629046
 8:05 pm on Dec 9, 2013 (gmt 0)

Russia, ok, although yandex is a force for good as far as I can tell (certainly an extra weapon against big G). I believe it obeys robots.txt if that is set up properly (there are several bot UAs depending on function). Do you also block yandex's North American IP ranges?

But what have you got against the UK and Ireland? And, for that matter, France, Spain, Portugal... We all have IP ranges in the 93 band, and I think the "good" countries outway the "bad"; as they do in most IP ranges, even including the APNIC ones.

I have to say I get far more "bad" traffic from the US than I do from Russia and Ukraine. I wish I could block the whole arin range, but I can't because customers come from there, paying and non-paying.

The correct way of blocking is to pick out troublesome ranges. Block all server farms as they are discovered (lots of places to start in this forum), block aggressive DSL ranges (including RU, UA, CN, US...) and keep a general eye on traffic.

Oh, and tell as many people as you can to prevent their computers getting infected. A LOT of bad traffic comes from botnets run across compromised computers owned by idiots and novices, mostly DSL home/office but a few server farms as well.

But of course, you will do what you want; as do we all. :)

lucy24




msg:4629079
 10:07 pm on Dec 9, 2013 (gmt 0)

I think in wilderness's specific case he deals in non-portable widgets, so it's easier to block non-ARIN ranges in one fell swoop.

Besides, there's a certain visceral satisfaction in being able to say
Deny from 8
;)

Angonasec




msg:4629226
 10:04 am on Dec 10, 2013 (gmt 0)

dstiles: No offence intended Sir :) my ancestry is UK IE too.

We simply block based on nefarious activities seen in our access logs, not nationality.

Lots of detritus witnessed emanating from the EU and Minor Islands, 93.

If they pass muster we gingerly Allow some UK and IE 93.

Yes, we block Yandex US servers as they are spotted. Though I couldn't give you a neat list, as you kindly provide, we just pile them in the dust-bin group in numerical order.

Same with the Sino US servers.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved