Forum Moderators: Robert Charlton & goodroi
Google had crawled some new pages (1 week) old, but not yet added them for me - MSN added then within about 24 hours. Yahoo isn't as fast as MSN either!
As for this thread being useless, surely this is the point of a forum? Google isn't the only SE, despite being the biggest. Given who owns MSN it is only natural to assume they have the resources to really take on Google over time.
In addition, in light of evident difficulties with Google over the last few months, I find it relevant and useful to make comparisons on the features we can actually observe and measure - especially given that much of the discussion about Google is speculative in nature. They are spectacularly bad at communicating anything, so any actual observations are welcome in my view.
Delete this thread - msn is not even 1% of the quaility of Google, why discuss it?.
Its like comparing a sunday league football team with a division one team!
MSN may well skim a few thin spammy sites quickly but it cant deep crawl a site like google bot can - currently im yet to find a quality authority site rich in content that ranks in msn that has more than 20% of its site content cashed by msn.
msn bot better than googles?, dont make me laugh!
This is a joke discussion thread.
Rich
Google only refreshed two pages that have links pointing back at me..
MSN sucks on my bigger sites. They don't even index everything, much less keep things fresh.
It might become slower when the database reaches the amount of google or more..
Any Thoughts?
It might become slower when the database reaches the amount of google or more..
Any Thoughts?
A smaller dataset is obviously going to be easier to manage.
However, you gotta love that search builder they have...
He's told, amongst other things:
Stop looking at your site(s).
Delete this thread.
Shouldnt this thread be in the MSN section?
Boy, those are some gems of wisdom right there.
His complaint is legit and it's happening with larger high PR sites as well.
Comparison to MSN aside, they've asked for sitemaps, we've given them sitemaps and yet the crawling is slower than ever.
Google has a problem and it should be a concern to all here.
Let's all do another cheer for "Big Daddy."
Google seems to do a better job of finding the specific page that gives good content on any given search term.
They are, but if you can't understand them, bad for you.
"His complaint is legit and it's happening with larger high PR sites as well."
It's not legit, it's nonsense based on anecdotal look at his own sites.
In my case some of my sites do well, others not so well in G, but for the several outdoors pastimes I enjoy (but don't have sites related to), I've found the quality of Google results has gone downhill over the past 2 years.
I don't really care how deeply MSN crawls some huge sites - they list the index pages of relevant sites on pages 1-3 of SERPs the way G used to, without too much auto-generated spam clutter. The MSN results seem familiar to me, and I get my info quickly. The G results have been frustrating lately (also familiar - from the bad days of Altavista or Lycos).
For some key phrases it's almost like they traded algos, as I remember MSN being the one susceptible to networks of spam and throwaway domains, now it seems like a Google thing.
Its not about how my site ranks. I will state some facts here.
1. Both googlebot and msnbot visit my homepage twice a day.
2. If I add new content to my homepage it does get spidered twice a day.
3. MSN updates it cache about 48 hours with the new information.
4. Google updates its cache 15 days later.
What I am saying, its not spam, etc... The bots are coming along to visit at least twice a day. What I am saying is why send a bot by if you are not going to update a cache? Make sense?
To add to this thread, I have no issue with serps, I rank top 10 for keywords in my niche idustry on MSN, Yahoo and yes I am going to say this GOOGLE as well. This thread is not about how sites rank, its about how fresh the data is.
BTW, Steveb... You mentioned to look at my site, please do. The site I put up December 2005, it did not hit the infamous "sandbox" and it is ranking top 10 in keywords. To add to it check the data centers for google and see its already getting page rank. I must be doing something right...
I actually think google created a monster with PR and, if you were brought in to evaluate their sustem, the first thing you might ask is what do you need this thing for?
The PR system as it is implemented is inherently prone to spamming/manipulation. Whatever holes google plugs, people find new ones (not surprising as there are a lot more indepedent webmasters out there than googleplex has techs).
So at any given time, google PR and hence serps are being heavily manipulated by practices designed to exploit holes.
My suspicion is Google devotes a ridiclous amount of resources to - as you say - chasing their tails: trying to keep the PR system from being totally overwhelmed by spammers. And there are signs the situation is getting worse.
As other pointed out, Google is handling spam WORSE, not better than MSN or Inktomi fed engines. MSN deal with problems, especially dup content and content theft, with no apparent fuss. Sites get PR0'd in google due to content theft, etc. and go down the tubes in Google serps. MSN seems to handle this with no problem, identifying and not listing most of the thieves.
I think Google senior management thinks their much promoted PR system is too important to scrap - not because it actually works, just because its a Google "trademark" - part of their product image. But I got to think some of them must realize it is ludicrously inneffective in doing what it is supposed to do: provide better search results.
Instead look at a wide variety of sites. MSN virtually never fully indexs anything over 50 pages, while Google will do so very often, but that really isn't the point. Just stop drawing conclusions about the Internet based on one website.
ebay
msn- March 2nd
google Feb 26th
(4 days difference)
foxnews
msn- March 6th
google- Feb 26th
(8 day difference)
ebay
msn- March 4th
google- March 2nd
(2 day difference)
Its not just my site per say, it is a lot of sites. Even big sites
For the vast majority of searches, it does not matter one bit to the *searcher* if the search engine only updated the index once a year.
I can only think of one search that I have done in the last 2 weeks where truly fresh mattered.
Just for kicks, I checked some pages that matter on some of my sites.
Personal website home page
Google Mar 5
MSN Feb 22
Large review site "new reviews" page:
Google Mar 5
MSN Mar 4
Small site devoted to specialty food item, home page:
Google Feb 2
MSN Feb 19
I think steveb is right here, it really depends on your site. Google REALLY like the review site, and the new reviews page is rarelky more than a day or so old. On my personal homepage, I think it was probably just the luck of the draw that Google won. Google has just crawled it more recently.
I do agree that some content never or rarely changes on some pages and should just be checked. That I totally agree on.
The point I am making is that googlebot is coming by and looking at the new content. It does spider it and it does cache it. It is just a lot slower of a process.
You also seem to be concentrating on the default page. One reason that google might hit your home page, and other pages that change fairly frequently is to look for new links instead of just having the goal of updating the index.
For google to update their cache, they need to work the page into the index, and then ship all their changes out to all the datacenters. If your page is changing constantly, then what it the point in doing that work when the cache is going to be out of date anyway.
If you do a litte research, you might also notice that the index and the cache do not always match. Many times the index will show a result for a search before that term shows up on the cached page.
