homepage Welcome to WebmasterWorld Guest from 23.22.179.210
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Newbie Spiders...
Laundry List of Unknown Spiders...
Eric_Lander




msg:407704
 1:53 pm on May 8, 2001 (gmt 0)

I have found the following spiders indexing (or at least visiting...) a number of my managed sites. Does anyone know where they come from?

ia_archiver
Zyborg 1.0
PolyBot 1.0
MLLXPC
Mercator-2.0
LinkWalker
WebSauger
Teoma (A new SE ready to debut?)
dllbot
architextSpider
OpenFind.Shark
Mc_Kalyan_21
ru_robot

 

Froggyman




msg:407705
 8:35 pm on May 8, 2001 (gmt 0)

Do a Site Search [searchengineworld.com]. There is info on this web site for most (if not all) the spiders on your list.

Jaf




msg:407706
 11:36 pm on May 9, 2001 (gmt 0)

Most of these are listed on the webbots page I maintain at
[jafsoft.com...]

That page lags behind the discussions here (I only just found this forum), but most of the ones you list are familiar to me

littleman




msg:407707
 11:39 pm on May 9, 2001 (gmt 0)

Jaf, welcome to WmW.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved