they must be shaking!
They're changing their shorts!
Google, add AskJoe's new pages post-haste!
Your reputation depends on it!
I agree that Googles index gets out of date too quickly (lots of missing pages before an update) but Freshbot has been extremely active recently for me, indexing many pages on some sites - even those without a particularly good PR.
> I've added several pages to my site that googlebot's been to but fresh hasn't touched
Freshbot can't get everywhere, or there would be no need for the deep crawl. The update process must take some time too, because of the many calculations in the algorithm. This pre-update has been strange though, with lots of unusual bot activity.
On your main point, there's always room for a new search engine and relevancy is definitely what will get the traffic if you ask me.
[hehe I need to type quicker]
Freshbot has hit about 3000 pages of my 6000 page site several times this month.
50-100 pages of my 500 page site get freshbotted nearly every day.
It must depend on how important your site is.
I am seeing updates ever 4 weeks, and I expect that within not too long (six months maybe) there could be some XML mechanism ala their froggle approach.
It makes sense to allow people who have updated their sites to submit the pages this way provided it is not at a cost or at least not a high cost, and does not affect rankings. Google essentially increases its spider capacity by allowing other to submit en masse.
Controls could be added to prevent abuse.
naturally for any webmaster or webmistress it would be great to have a real big se like google with a real-time index etc. pp. - but this is not possible at all. that much stuff changing in the net all day long...
i don't know wich company can startup and crawl the web every day to have every page indexed / checked. this will be so much bandwidth and storagespace unable to handle. you can't make a copy of the net itself i think ;).
>freshbot never really indexes anything but a home page
Well I don't know where Google gets all its news stories from then!
All I'm saying is that a 4-6 week gap between updates makes it searches less relavant than they could be - don't you guys think that someone else could come in with a simlilar algorithm and quicker update to surpass them?
What if a search engine provided the same relavant searches but updated its index weekly? or bi-weekly?
Wouldn't this be valued much more?
I have a new site this month and im waiting for the update to get some PR. the freshbot has picked up the new site as ive added a few links to it. Im getting hits from google already and nobody else. For me that tells me they are still head and sholders above anyone else!
Copying/indexing the net is a difficult task indeed. Whoever figures it out will be banking like Trump!
|All I'm saying is that a 4-6 week gap between updates makes it searches less relavant than they could be - don't you guys think that someone else could come in with a simlilar algorithm and quicker update to surpass them? |
Sure, all I need is about 30k servers and 2-10Gbps multi-homed connections. I'll kick google's behind then :) He-He.
>you can't make a copy of the net itself
I really like that line ;)
My PR6 site gets "freshed" several times a week, on average 250 pages are crawled. If you want 48 hour updates there's always Inktomi PFI!
MSN is providing fresher content right now. Nobody has to come along and do it.
now if msn only dumped that lame horse looksmart. Then it would be inktomi and I'd be happy.
Who exactly would be in a position to exploit this 'weakness'? - only one that comes to my mind is Google!
Overture-Fast and Yahoo-Inktomi, they're not picking teams for nothing!
> you can't make a copy of the net itself
Reminds me of the giggle about when a young lad saw a set of
World Book encyclopedias on the book shelves at grandma's house ... "Wow, you printed the whole CD?!?!"
Not only am I eager to see new results, but it does hurt business searching for new avenues when its the same ol' crap that I've seen 7 weeks ago!
Who has fresher content? Google has more of my pages as fesh content than any of the other engines has in their regular index. And most of the pages that the others have in their index are months old.
Most of the time, the searchers really don't care how old a page is. The majority of the useful pages on the web are still plain old .html for a reason, their content rarely changes.
does anyone knows that there are still people doing their job in web researching to pick out the latest infos relating to someones other buisness as a service? shure these people use google but they don't stop only because what they are looking for is not in googles index...
i mean it's only a search engine not a research engine ;)
askjoe, I like your point of view. Sticky me when you have ready that kind of "special-for-anxious-webmasters search engine"
Googles weakness is that they say "No this, no that" in their TOS. Used to penalize for it, but currently do not.
The problem being, do you join the Spammers and follow suit, or wait and hope Google practice again what they preach?! This has been going on for 6 months and Google use the same "We have a few surprises lined up" replies, but I think that is rhetoric to stop others following suit until they resolve either the technical or legal issues / problems that made them strip certain anti-spam scripts from their algo in the first place.
I completely disagree.
I can add a page with a brand new product and sometimes make a sale within days. Nobody, I repeat nobody, can touch that. It shocks my employer when it happens so I warn that when we add new products "you better get stock because you never know what will sell, or how quickly".
As usual I will defend Google.
I'm not saying in general MSN is fresher than usual but if freshbot is not a regular visitor... Inktomi is indexing pages that have been added since Google's January crawl and has been for three weeks.
For some reason, i'm not getting fresh tags despite about 80 visits from freshbot this month.
It all depends on how often freshbot drops by. Half one of my sites has a new date of March 3 including pages I just put up within the last few days. So freshbot must have been around today.
I've been wondering how often you have to update to keep freshie visiting that regularly. I know you don't need to update daily because I don't update daily. Might it be once a week or once a month?
on the topic of making a copy of the net and putting it on CD....
funny one of my sales guys asked about this in 2001. he wanted a copy of the site on CD so he could....show...clients....the....application....
The whole engineering team was like...dude if they don't have the internet then should they really be using our product...
You sound just like me.. this is treacherous for anyone who wants things to happen yesterday.
As for Msn or anyone being better and more up to date than Google..NO WAY!
I have a new site that was long overdue for the internet. At first I was having problems dropping in and out of the index.. but as long as I add to my content a little bit everyday or every other day pretty much all of my pages stay in. I am very appreciative of Google for giving me
close to 1500 uniques a day and rising with no page rank.
Where else.. even in the real world can you go and receive that much traffic without paying a dime?
MSN.. Alta Vista, etc. have sent 0 traffic my way.
| This 41 message thread spans 2 pages: 41 (  2 ) > > |