homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

new spiders?
well - at least new to me

 3:01 pm on Jul 18, 2000 (gmt 0)

I've been hit by 'The Informant' and 'NetAnts' everyday for the past week (sorry, I don't have the IPs). Anybody know who these visitors are?



 3:02 pm on Jul 18, 2000 (gmt 0)

got NetAnts - it's a download manager.


 3:58 pm on Jul 18, 2000 (gmt 0)

The Informant is a bot that will crawl listings of the search engines and report when there are any sites that appear. It is a university's CS project. informant.dartmouth.edu [informant.dartmouth.edu]

"The Informant is a FREE! service that will save your favorite search engine queries and web sites, check them periodically, and send you email whenever there are new or updated web pages."

It is actually a nice tool.


 10:20 pm on Jul 22, 2000 (gmt 0)

Not seen this before:

karl-spider/0.2 (Under development. Contact sassenbe@cs.ucc.ie Not affiliated with AltaVista in any way.)

What a great UA!

Mailed them already, will let you know.

Edited by: NFFC


 8:46 am on Jul 28, 2000 (gmt 0)

Here I thought I was the only one that knew about the Informant. I find it is a killer tool. I use it all the time and have toyed with a similar rank tracking system for myself. I've thought about putting it on sew, but I got enough se raiders and emailers going on now. New domain time for some of these tools...


 4:40 pm on Jul 28, 2000 (gmt 0)

Only problem I have with Informant's spider is that it is not identifing itself, in fact I've 403 it on server level because it started getting very aggresive.

Anybody familiar with griffon/1.0 (griffon@super.navi.ocn.ne.jp)?
Its been knocking on my door for quite some time and I decided to let it in as it seems to resolve to some sort of japanese SE/directory, so far its behaving itself but I did not see any surfer visits from navi.ocn.ne.jp in the logs yet.


 4:43 pm on Jul 28, 2000 (gmt 0)

Oops, I think I mixed up Informant with the Cyivillence (sp?) thing, they are the misbehaving kind :)

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved