Forum Moderators: open

Message Too Old, No Replies

Symfony BrowserKit

         

keyplyr

8:43 pm on Feb 23, 2018 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



UA: Symfony BrowserKit
Protocol: HTTP/1.1
Robots.txt: No
Host: triolan.net (Ukraine municipal)
37.57.221.0 - 37.57.221.255
37.57.221.0/24
Parent: Tov Bank-inform (Ukraine ISP)
37.57.0.0 - 37.57.255.255
37.57.0.0/16

This UA can be run from any server using any IP range. It's basically an HTTP client that simulates a browser. Most often the UA string is edited to appear human and undetected.

Other UA variant: [webmasterworld.com...]

lucy24

9:58 pm on Feb 23, 2018 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



simulates a browser
How lifelike is the simulation? Do supporting files get requested suspiciously fast*, after a reasonable delay, after an unreasonable delay, or not at all?


* Paradoxically, that's what made me look askance at the latest Chromeless [webmasterworld.com]. The last straggling requests ought to come in a second or two later, not all at once.

jmccormac

11:05 pm on Feb 23, 2018 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Looks like a scraper. Banned like a scraper.

Regards...jmcc

keyplyr

11:49 pm on Feb 23, 2018 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



How lifelike is the simulation?
The operator can edit the UA string to say anything. You've likely been hit without knowing it. The requests could be innocent (server admin using this script to browse sites) or it could be scraping (as jmccormac mentioned.) The requests can be controlled to GET all files on a document (like a normal browser does) or just set to request specific file types (scraping behavior.)

Personally, I've blocked it, & the other variations, for a while. However, if the UA is modified to spoof a bowser, there's usually not any other indicators to signify it isn't a real browser. Header fields could be normal.