homepage Welcome to WebmasterWorld Guest from 54.167.96.124
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Help needed
adamcmoran




msg:405909
 1:59 pm on Sep 6, 2000 (gmt 0)

Could some kind person decipher this for me as I have not a clue

Slurp
Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; fs_ie5_04_2000i; MSIECrawler)
Slurp.so
FAST-WebCrawler
Scooter
Gulliver
Lycos-Proxy
DIIbot
Aranha
Lycos_Spider_(T-Rex)
Mozilla/4.0 (compatible; MSIE 5.01; Windows 98; MSIECrawler)
Kenjin Spider
Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; Site Server 3.0 Robot) certbytes
SearchExpress Spider 0.99c obj
InfoSeek Sidewinder

Adam

 

eljefe3




msg:405910
 2:44 pm on Sep 6, 2000 (gmt 0)

Looks like your browser log. This identifies who has requested pages from your site.

littleman




msg:405911
 4:04 pm on Sep 6, 2000 (gmt 0)

And a nice one at that!

adamcmoran




msg:405912
 4:22 pm on Sep 6, 2000 (gmt 0)

I realise that it is my browser log
Can someone identify the spiders if possible?

Adam

oilman




msg:405913
 4:40 pm on Sep 6, 2000 (gmt 0)

Here's a quick pass at what I can pull out of my brain on the spot:

Slurp - Inktomi
Slurp.so - Inktomi (International)
FAST-WebCrawler - Fast - Alltheweb.com
Scooter - AV
Gulliver - Northern Light
Lycos-Proxy - pretty obvious
Lycos_Spider_(T-Rex)- pretty obvious
InfoSeek Sidewinder - pretty obvious

Hope that helps. Anybody able to fill in the gaps?

adamcmoran




msg:405914
 4:45 pm on Sep 6, 2000 (gmt 0)

DIIbot
Aranha
Kenjin Spider

I am especially interested in what these three are as I don't even recognise them.

ADAM

tedster




msg:405915
 1:46 am on Sep 7, 2000 (gmt 0)

I used IXQuick to search on these three for you. Here's what I came up with.

DIIbot belongs to a company called Digital Integrity [digital-integrity.com] (The link goes to their spider info)

Aranha is the name of a Spanish language get-paid-to-surf site. I am guessing that this is their spider, but I don't know for sure.

Kenjin is an information technology product from a compnay called Autonomy. Their purpose is to help an individual bypass dependency on the commercial search engines. That sounds intriguing, but I didn't have the time to explore any further.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved