Forum Moderators: open

Message Too Old, No Replies

semantic-visions.com crawler

just an HTTPClient

         

keyplyr

7:39 am on Mar 8, 2017 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



UA: Mozilla/5.0 (X11; compatible; semantic-visions.com crawler; HTTPClient 3.1)
Protocol: HTTP/1.1
Robots.txt: No
Host: hetzner.de
144.76.0.0 - 144.76.255.255
144.76.0.0/16

Requests first came from a server based web browser:
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.4) Gecko/20060406 Firefox/1.5.0.4 (Debian-1.5.dfsg+1.5.0.4-1)

Then the user switched UAs & pretended to be a branded crawler, which in fact is just a simple HTTPClient with a modified UA string.

Nice try

lucy24

6:30 pm on Mar 8, 2017 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Firefox/1.5.0.4

Heh. I'm pretty forgiving about outdated browsers, but that one would run slap into the bad_agent environmental variable, never to recover.

keyplyr

7:51 pm on Mar 8, 2017 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



It's not a Windows version though. Linux browsers have build numbers far behind that of other OS.