So I was assigned a project to research for information different web crawlers that can be used. I'm supposed to compile a list with their different features, designs, specifications, etc (I don't even know what exactly I would look for in detail about Web Crawlers). Could anyone provide some information or maybe a list of specific Web Crawlers that I could research to get me going?
Thanks!
wilderness
4:13 pm on Jul 17, 2008 (gmt 0)
More than enough to get you rolling.
(Archive links provided to assure longevity in Webamster World archives)
