homepage Welcome to WebmasterWorld Guest from 54.227.25.58
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Web Accelerators
Characteristics?
vortech




msg:3252130
 6:40 am on Feb 14, 2007 (gmt 0)

Can anyone give me any tips on how to tell the differance between a bot or if it's just a browser with a web accelerator?

What I'm seeing is a regular UA but it doesn't always get images. It starts out normal but ends with no images. It does this at a "human rate" and doesn't take very many pages. I see this from different IPs and they always go to different parts of the site, just like normal.

Another normal UA I see seems to be multi threaded in that it gets 2 pages in the same second. Sometimes the pages are the same page called twice and other times it calls default and the second page called is different, but at the same second.

Clear as mud I know.

Are these web accelerators? Or bots or browsers with the images disabled or am I just looking at log files too much?

Vortech

 

wilderness




msg:3252975
 1:44 am on Feb 15, 2007 (gmt 0)

vortech,
most web accelerators (or similar tools), although perhaps NOT identified with an appropiate UA, will come from a reputable IP range of either a search engine utilizing some type or tool or a software that is exclusively for acceleration.

There's some old threads on a few of these

[google.com...]

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved