Forum Moderators: bakedjake
[grub.org...]
It appears this will be able to not only refresh the hosts index but "crawl" the web and refresh the web index on a daily basis.
Very interesting process much like Napster and Seti.
Certainly will give apowerful and cost affective edge to Wisenut.
You can enable local crawling (your sites) and dissable remote crawling (other sites) in the preferences in the software. Obviously they don't want you to do this, but you're still doing them a favor....
By spidering your own sites, compressing it up and sending it to them, you are sparing them the bandwidth to crawl your site. You'll also be more inclined to make sure all your content is properly spidered and fresh. So they still win.
Maybe if this were to catch on and everyone spidered their own content, there would not be much other work to pass on for other members to spider anyways.
Kind of like all of us calculating/sending in our tax refunds like good citizens vs. getting the IRS to go on a hunt for our heads and do it the slow and painful way :) Spidering your own content and sending it in could end up becoming the norm.
I spoilt the party by breaking the news from getting the information from the LOOK annual report.Its amazing what you can find what comapnies are doing in annual reports/worth the look sometimes.
Judging from what Kordless has said i would think the 18th will be a nice date for others then on here to find out,just say hard to hide from us.
And from what i gather,what we are seeing now is not what they will be rolling out as the full version of this,as this is in beta.This will really heat things up in the search engine freshness area and if Looksmart can pull this off with Wisenut technology supporting it,could be a winner.
Born2 brought up some excellents points re the future of how this could be implemented too.
To have your clients sites refreshed daily I would have thought would be in yours and your clients best interests. I see it as a two way street. True, you are providing bandwidth according to your limits and not "Grubs" appetite. It is entirely up to you.
You are completely right that this would make sense for an SEO. But I did not think of SEOs; I thought of the normal users - and they are the ones who have the sheer number that a project like Grub needs.
And for the technical aspect of misleading the Grub crawler: This program runs on my computer. It's very easy to manipulate domain name resolution on my local system. (using my own DNServer or just editing my hosts file) No checksum can help here. I'm not saying that it is easy to spam Grub this way but people are doing crazy things just to get a better PageRank. So Grub should be aware of its weakness.
IMHO I think Grub really shoulda been something targeted to only run and monitor a server's (or networks, like a blog ring or some such) web documents, and then webmasters would be able to write-up a fancy front-end showing their latest updates (such as new products) and other updates wherever and however they please. A local system, and not so much a system that says "run this, and don't benefit," which, in its current stage, is exactly what it's screaming.
Very good to see, let's push this baby to the max! Google would be quite interested in how far this chart can go I would think.
This Grub.org thing is interesting, what say you all?
~Hollywood
P.S. Keep pushi'n
+ what about their developers? will they be working for looksmart now?
Note the latest version of Grub was released today with an auto update feature for the host.
On how much LOOK paid....it was a scrip deal and its in the annual report too.
Please explain
http://news.com.com/2100-1032-993591.html [news.com.com]
In January 2003, we acquired substantially all of the assets of Grub, Inc., a developer of
distributed computing software which allows community participants to assist in the development and
updating of a web search index. We believe that by incorporating a distributed computing solution
into our systems and processes for updating our search index, we may be able to achieve substantial
gains in the freshness of the index and cost savings over the long term.
i didnt follow this discusion before - any changes since last post (month ago)? is it the breakthrough like they say on grub page?
