homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

Incy Wincy?
Looking for some information on this one...

 8:17 pm on Sep 4, 2001 (gmt 0)

Appeared in logs:
IncyWincy data gatherer(webmaster@loopimprovements.com,http:

I found that reference to a visiting spider, andd was just wondering if anyone could give me some dirt on them... I emailed the webmaster to see what they're about, but the message got sent back. Any information would rock.

~ Erock ~



 9:49 pm on Sep 4, 2001 (gmt 0)

Had them in my logs also, visited their site. The concept basically seem to be to first search odp listings, then show which of the those listings has an inbuilt search facility, then displaying that search facility under their own header for further searches.
"Unleashing the invisible net"? Well not exactly.


 4:23 pm on Sep 5, 2001 (gmt 0)

It looks like IncyWincy is for sale to be installed on your own server. If it's for sale, then possibly your competitors could have IncyWincy spiders of their own?


 4:51 pm on Sep 5, 2001 (gmt 0)

It hasn't hit me. Do you have the IP of the bot?


 6:50 pm on Sep 5, 2001 (gmt 0)

littleman, I will access the logs and look it up as soon as possible and let you know...

As of right now, I'm waiting on an admin of the server to log off timbuktu.

Any thoughts as to whether they should or should not be allowed to index a site?

Just curious, and much thanks again to all.

~ Eric


 7:01 pm on Sep 5, 2001 (gmt 0)

Okay, I did some homework!

IP Address was consistently:

However, in two entries, I found that the user agent was identified as "IncyWincy Data Gatherer" and later "IncyWincy Page Crawler"

Isn't that odd to see the same IP, with two identification names?

Hope this helps littleman.

~ Eric


 7:04 pm on Sep 5, 2001 (gmt 0)

I've also seen Incy Wincy spidering from


 8:02 pm on Sep 5, 2001 (gmt 0)

Looks to me like it is a product that will create a locally hosted database which is created from spidering the ODP.

The IncyWincy Search Engine runs on your server and is available for all popular server platforms. Every week an updated database is available with the latest content from the Open Directory Project. The database is supplemented with Invisible Web information on each site that has forms for searches and other functions. These forms surface as a set of search engine links you can query within the context of search results and category navigation. The search engine links are the ones found on the sites present in the search results or category site list. Such Invisible Web functionality greatly increases access to web properties in one place without having to ferret out the information all over the Web.


Seems like a legitimate product to me.

Thanks for the IPs.


 4:47 pm on Oct 14, 2001 (gmt 0)

209-87-236-249.ottawa.storm.ca "IncyWincy(http://www.look.com)"

Ignores robots.txt.


 9:47 am on Oct 15, 2001 (gmt 0)

I've just had a play about on [incywincy.com...] (the live demo of the loopimprovements service). It looks like they are trying to create a new service based on the ODP and using site searches built into the websites. A normal search displays ODP results plus a list of those sites IncyWincy has found with built in site search facilities.

In theory, you would search in the same way as you would with the ODP, and ODP results are listed. The IncyWincy search results will also list all sites within those results which have site search facilities built in. You can then (in theory) search those sites directly from IncyWincy.

One sight flaw though ... it doesn't work very well. I checked 5 sites that IncyWincy claimed had site search facilities. 3 of those were whois boxes (not site search), 1 had an error, and one worked but displayed site search results in a foreign language (not much good to me).

Will this ever be useful ? Maybe. It needs a lot of bugs to be ironed out first though. I can't really see it succeeding long term unless some of the big engines and directories go bust. It seems a very long-winded way of searching for relevant results when Google throws them up in a split second.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved