homepage Welcome to WebmasterWorld Guest from 54.211.34.105
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
lynx robot
too much information?
lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4475776 posted 4:14 am on Jul 15, 2012 (gmt 0)

Query: If I didn't have external evidence*, would it be obvious that this is a robot?

Lynx/2.8.5rel.1 libwww-FM/2.14FC SSL-MM/1.4.1b OpenSSL/0.9.7d-dev

For comparison purposes I visited the same page myself in Lynx and checked the log. It says only

Lynx/2.8.6rel.5 libwww-FM/2.14

Is this one of those stereotypical police scenarios where you sit back and let them hang themselves? It should have stopped at 2.14FC :) (Interesting that real Lynx doesn't send any information about computer or OS. Never knew that.)


* It visited the same page twice, almost concurrently-- within the same second in logs-- using two different referers. I will concede, Don, that this is as close to physically impossible as you can get. And then proceeded to the home page in less time than it would have taken a human to find or type the link.

 

incrediBILL

WebmasterWorld Administrator incredibill us a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



 
Msg#: 4475776 posted 5:18 am on Jul 15, 2012 (gmt 0)

Just because it might be the real Lynx doesn't mean it's not being driven to read pages by a bot script, pretty clever really.

[edited by: incrediBILL at 5:55 am (utc) on Jul 15, 2012]

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4475776 posted 5:53 am on Jul 15, 2012 (gmt 0)

lucy,
There's an online tool that generates that UA.
Anybody may use.

Lynx Viewer [delorie.com]

The intent is to see the underlying-HTML in the same manner that bots do.

I use to use the thing on my own sites frequently some years back, however had the UA denied when I wasn't using it.

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4475776 posted 5:59 am on Jul 15, 2012 (gmt 0)


Ha, that tool doesn't work for my sites.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved