Welcome to WebmasterWorld Guest from 220.127.116.11
Forum Moderators: mack
For most of my sites, msnbot has crawled almost all the pages, but only 5% shows up in the beta. Same must hold true for other sites as well, as I believe.
I hope that they release a full version of the beta when they launch next week, if they do.
Edit Reason - Spelling correction
[edited by: Imaster at 9:25 pm (utc) on Jan. 14, 2005]
How in the hell is it on par with Google if it doesn't even have 10% of pages crawled for a majority of the sites. I seriously can't believe MSN Beta has over a couple of billion pages indexed.
[edited by: Imaster at 9:36 pm (utc) on Jan. 14, 2005]
However, given that the initial users will be the users who were clueless enough to keep using msn in its yahoo version, I'd say that group of searchers aren't going to notice a lot of difference, so in a way it's not a very risky move. Of course I don't see how they'll get new searchers in yet, maybe they'll improve it live?
We'll see, I'm still trying to figure out the recent algo tweak they did, distinctly different type of results, quite a significant difference from what I'm seeing, though some stuff hasn't changed. We'll see.
I am happy for one not for profit site I do, we're top there for a term I've never been able to get them in with the others, that should help a bit.
I hope the little bird giving inside information is more accurate than the last little bird that said to expect a big google update last fall. Probably is, but we'll see. I also welcome more pressure on Google, I think they really slipped majorly this last year, from being a truly admirable market leader to just sort of plain old product that isn't really going to wow anyone.
Now if Yahoo would just start putting some real resources into their search, they have exactly the same indexing problems people are seeing with MSN beta, incomplete, dropped pages, etc. It really looks like currently no search engine is actually able to handle today's web, at least not in the way google handled it from about 2000-2003. That's complete, total indexing, all the time. Of all pages.
But I guess indexing that many pages get's pretty hard to do.
I installed a search engine and spider on my site and just around 5,000 pages was hard on it. I can see how hard it would be to index and search 5 billion would be. And on top of that, get good results at the same time.
Yes, it does. But considering how many of those are spam, the real trick is to start figuring out more creative ways to get rid of it. that means putting down the programming tools, and putting on the thinking caps. And maybe the solution won't be only a programmed one. Spammers know this works, it's how they do it. At some point search engines will have to also learn this.
But that's their job, that's what they do. When google came long nobody was indexing 1 billion pages, and they didn't really have much problem until they hit 4 billion. Maybe it will take a new company to come along who won't have problems with 10 billion, or 20. If these guys think that they are worth a few billion dollars each, then they should be able to figure out the problem. Of course, I don't think they are worth that, and I'm starting to doubt that they are able to figure the problem out.
Rather than check my own results, I decided to actually "go shopping". I found mostly spam and affiliate sites on the first page of results for my first attempt. Very frustrating. Pretty good results on the second page though.
Then I decided to "go travelling" ... same thing pretty much.
Then I checked some of my own keyword phrases and although there are a lot of sites using every SEO trick in the book ranked at the top of the pack ... they are at least relevant sites.
I'm not terribly impressed so far, but hopefully they will improve with time. I'll keep an open mind.
The spam really does bother me though and I think it is going to be pretty easy to manipulate results. Content doesn't seem to have a whole lot to do with rankings ... but titles, headings and URL's sure do!
Bye Bye Google!
Unless you are the top level business guru of the top investment firm in the world then I don't think any of us are qualified to speculate accurately on how this will affect Google. Yes, I think Google is an awful search engine now, can't find a damn thing! BUT - being good at something isn't what makes you top ...
It's only the Webmaster community that cares (or even knows) about the upcoming change. 95% of search engine users will have no clue and will continue to use Google. Our website traffic will increase due to the change but I can't say that this will impact users for a very long time.
Another issue with MSN beta is that most of the sites in almost all topics have only around 5-10% pages crawled as compared to Google.
Personally I have found the MSNBot to be the hardest working and the index to be the quickest updated. Within 24 hours of me launching a site, they had over 500 pages of it showing on Beta. Google still has 1 (the main page) after seven days.
I have found that MSNBot has a certain "dislike" for dynamic pages ... strangely, this includes those that use mod_rewrite to appear static ...
But considering how many of those are spam, the real trick is to start figuring out more creative ways to get rid of it. that means putting down the programming tools, and putting on the thinking caps.
It's so strange isn't it? The majority of "PHD's" (no offense to the better educated amongst us) never seem to be able to think outside their box - they tend to think that programming will always win when it is not programming they are fighting against, it's human creativity and that's something you can't learn in a school.