homepage Welcome to WebmasterWorld Guest from 23.22.128.96
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Timestamp in useragent string?
trintragula




msg:4640460
 8:18 am on Jan 27, 2014 (gmt 0)

Someone showed up on our forum recently with a time stamp in their useragent string, so every request has a different useragent.

I've not seen that before... Looks like a Chrome UA with perhaps an extension.

Anyone know what this is?

 

dstiles




msg:4640561
 8:13 pm on Jan 27, 2014 (gmt 0)

Might help us to know what the UA is.

trintragula




msg:4640569
 9:10 pm on Jan 27, 2014 (gmt 0)

Here's an example:

Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.220 Safari/537.6 mpVrf2ZWrek6Mi9ZYzIwPJvc358= 2014-01-26T16:41:12

(it's possible a few chars have been clipped off the end here - I had to rescue the data when the log expired.)

lucy24




msg:4640582
 9:53 pm on Jan 27, 2014 (gmt 0)

I saw this once from a Polish robot. I think the idea is to bypass UA-based blocks by having a different UA string on each request. The one I met came with a further quirk: logs tell me the robot's clock was several seconds behind my server's.

:: detour to search ::

Here's [webmasterworld.com] the resulting discussion. More recent than I thought.

trintragula




msg:4640602
 11:00 pm on Jan 27, 2014 (gmt 0)

In this case I'm thinking it's some kind of website downloader browser plug-in/extension with a humanoid driver, mostly because they succeeded in signing up on the forum with it.
The IP is in Australia, which is plausibly okay on our forum.
(They also activated the account and logged in later with a mobile UA that has no timestamp.)
The timestamp format is different from the one Lucy saw, so this may not be the same animal.
I'll keep an eye on them.
I've not previously added a trap for rotating UAs, but I'm thinking about it.

A year is a long time on the web...

lucy24




msg:4640626
 1:33 am on Jan 28, 2014 (gmt 0)

The IP is in Australia, which is plausibly okay on our forum.

Come to think of it, I've never met an Australian robot. At least not to remember. And, when all is said and sifted, most robots infesting US-based sites ... come from US-based server farms.

For people who don't have time to pursue links: Mine, from just over a year ago, was a fully humanoid Firefox-based UA, apart from the appended timestamp. And then the discussion veered into incrediBill's header-checking routine. I knew I got it from him, but I would never have known that this was its home thread.

dstiles




msg:4640853
 9:55 pm on Jan 28, 2014 (gmt 0)

I tried running mpVrf2ZWrek6Mi9ZYzIwPJvc358 through a base64 converter but it didn't make sense. It's possibly (probably?) and encrypted string saying something like "Date/Time". Not entirely a joke: I regularly get attempts to authenticate on my POP3 server using the string UGFzc3dvcmQ6 which is base64 for "Password".

The rest of the UA seems valid.

trintragula




msg:4640870
 11:19 pm on Jan 28, 2014 (gmt 0)

I'd wondered about Xing it out before posting it here in case it's a beacon of some kind - but maybe that's a little too paranoid. :D Probably the timestamp would be good enough for that anyway.
Perhaps it's an encrypted node id to avoid massively parallel scrapes from being stopped by counting UA coincidences.
Who knows...

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved