homepage Welcome to WebmasterWorld Guest from 50.16.130.188
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
spider database
PeterB




msg:397509
 11:45 pm on Mar 18, 2000 (gmt 0)

Where is the best place to get the most complete, up-to-the-minute spider IP,host, and agent information?

The list on this site is an excellent resource, but it seems as though it is missing some entries. People post to this list about new bots they discover, but are those entries added to the master IP list?

I would love to have a FTP accessible, delimited list that I could automatically retreive and add to my cloaking scripts...

Any ideas?

Thanks,

Peter

 

fantomaster




msg:397510
 9:51 pm on Mar 20, 2000 (gmt 0)

Peter: please contact me by email and I'll point you to our commercial fantomas spiderSpy(TM) service.

Barry Geiman




msg:397511
 3:48 pm on Mar 23, 2000 (gmt 0)

As an IT Recruiter I am always trying to find new ways to search the internet, whether it be web robots, web ferrets or web spiders. I consider myself to me very internet savvy, but I have not heard of a "spiderspy", what is it and how does it work, plus where can I get additional information to read up about it ?
Barry Geiman <barry.geiman@staffingresources.com>

fantomaster




msg:397512
 4:13 pm on Mar 23, 2000 (gmt 0)

You are free to contact me by email as well. <img src="http://www.searchengineworld.com/Images/happy.gif" border=0>
But ok, since you're asking, I guess Brett won't object to my flaunting our
commercial service:
The fantomas spiderSpy(TM) service is a search engine spider IP database
(to the best of our knowledge: the world's most comprehensive).
The data offered is crucial for IP delivery and cloaking.
It's not restricted to US based engines either. Updated every 6 hours and
accessible to paying subscribers. 'nuff said.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved