Welcome to WebmasterWorld Guest from 188.8.131.52
Forum Moderators: mack
At the expense of repeating myself, I'm going to repeat myself. :-) Here are the guidelines I'm trying to follow when I post here:
I won't discuss the problems of particular sites in any public forum. As long as I can handle the volume, I'm happy to get "sticky" messages, though.
I also won't discuss competitors here -- they have their own forums, and I figure they already have enough people talking about them. :-)
I will not preannounce anything specific here; I might say "we're working extra hard over the summer to fight Spam," but I'm not going to post something like "look for a giant spam-killer release on April first!" After a release goes out and someone notices it, THEN I'll comment on it. If no one notices it, it probably wasn't worth talking about anyway.
Finally, I'm not going to reveal any secret Intellectual Property that would help someone build their own search engine or hack ours. Most of it's hard to explain anyway. :-)
[edited by: engine at 1:54 pm (utc) on June 5, 2006]
What I have seen of it over the last year is it has fast indexing of new information, making it a useful tool especially for research.
Seems to have got a major portion of the SPAM out of the index and when I say spam I mean keyword stuffed documents and black hat seo tricks.
Seems to be making good information easy to find.
I have a feeling these are part of the vision but .. is there anything that I have missed that will play a major part in it?
I guess if you want unpopular sites that nobody cares about then MSN rules.
msndude : If that is going to stick for this part of the world then certainly over here I think people will be happy, I always try searches outside of our own area of interest for some general search and those ninemsn results come up with lots of local content and direct websites as opposed to just sites with a few words or a link.
But seriously the more I query MSN since this update the more I feel that these guys won't make it in my lifetime, and I am still quite young and healthy.
One word queries, even some of the most obvious ones are, across the board just so wrong.
Is it that complicated to find real authority sites?
Is it THAT complicated to filter subdomains, hyphenated domains, blogs which florished on top of MSN during this update?
I don't pretend to know better than MSN engineers, I would love to see a more equitabl market share in the search world, but what I am sure of is that if my being a CEO at MS search there would be a few that could worry about the next paycheck. :)
This being said, it is hopefully a moment of transition - MSN dude knows better I guess.
On the other hand, MSN should not put too much emphasis on keywords in url either, which seems to be a major problem in the latest update. Many people use their own company names as domain names, they can't insert keywords into their domain names just to get high ranking.
But going forward i really recommend 2 basic fundamental relevancy tests included with your testing procedure.
1. Is my own website, search.msn.com found anywhere in the results for common phrases?
msn search needs to be found somewhere for its most basic phrases/keywords: internet search engine, online search,internet search, search engine (this phrase is found, but msn beta domain appears?)
2. The place/webmnasterworld i am telling everyone about the new results/or news about our search engine, is WebmasterWorld found for common keywords/basic terms?
I think these 2 basic fundamental relevancy tests have to be in there to call an update a true success (in my opinion)... as far as relevancy goes.
Think of it like the phonebook. a search engine is basically a search of a phonebook of websites in a similar basic fashion. Verizon hands out a yellow pagse phonebook, but verizon itself isnt listed under the Telephone Company section. Likewise verizon announces their new updated phonebook in the newspaper, but the newspaper itself isnt listed anywhere in the phonebook where you would expect. At that point, something is off when the telephone company itself isnt found and the place it is announcing/discussing new features of the phonebook isnt there either.
actually speaking of telephone here is another good test. Search msn for: telephone company
no at&t, qwest, bellsouth, verizon, etc..
i expect at least 1 or 2 to show up somewhere for a basic query.
2 basic tests, Is my own domain search.msn.com found in the serps for typical / common keyword phrases and is the site I am announcing updates/news on WebmasterWorld, found for somewhere reasonable for its typical phrases.
i dont expect you to revert back to the old results either.. i thought they lost some relevancy/took a dive back in the jan/feb update.
but hopefully in your experimental sets you can play around with it and these 2 basic tests will appear somewhere within the first couple of pages. Keep at it and i defenitely appreciate your response and listening to our feedback and suggestions.
Why don't we start a collective [open source?] search engine project...with the goal of being taken over by one of the three majors wanting to improve their algo?
Please do some analysis of the current index.
1) Pick 10,000 most competitive keywords (you should not have any trouble in this department)
2) Collect the first 10 results for each
3) Query the results for URLs with keywords in the domain name.
4) Examine the URLs with keywords in them, and the overall density of such URLs
5) Run the same against old index and major competitors. Compare the results.
Iím sure with the use of wonderful Microsoft SQL Server Reporting Services it won't take you long to put together a report like this. Once you visit a few URLs from the list you may become a bit less excited about the new "net".
My overall impression is that you did remove lots of spam but introduced different kind of spam at the expense of relevancy.
I loved the old A9 search results (after they went with MSN), but definitely won't be using the new ones. One search I did brought up an entire top 7 consisting of subpages of one site, and the next 3 in the search results were pages from another single site. Both of these sites had keyword rich domain names, and both had always ranked below #50 before the update (How do I know? - because I fanatically check search results across several search engines for a couple of terms daily)
I can assure you these new pages iam putting up are not spam or black hat. The only thing I can think of is msn doesn't want to index anything new comming from my domain or there has been a huge delay in getting pages into the index.
Let me know if you can.
DEEP CRAWL OUR SITES, DEEP CRAWL OUR SITES, DEEP CRAWL OUR SITES!
Sorry but until your search bot is effective as Googles or Yahoos at deep crawling and indexing total site content you simply wont be able to get enough data to even start to put together half respectable SERPS.
The only sites you currently index well are bloggs and sites with less than 500 pages. Anything 10,000 upwards and you can count msn out.
Sorry but your search bot technology cant currently match that of gigablast!
Garya - First off, this is a completely idle threat and ultimatums are never a good idea. Personally, when poor results are showing this is perhaps the best time to advertise on MSN.
Give the guys a chance. They didnít buy a search engine, they are creating their own. Theyíve been doing this for what - a little over a year now? Yes, I understand this is MSN and theyíve got deep pockets. But all the money in the world cannot replace experience and Google and Yahoo have a lot more experience tweaking their algos.
And I donít think MSN cares about the complaints, they are looking for suggestions.
Yes msn have been in the market for a year, but they dont help their own cause by putting out statements saying what was it now?, "we will have the most relevent search results within 6 months" etc etc
Sorry but they are miles away:-
1. They havent learnt how to get the search bot to deep crawl yet
2. They are still messing about with outdated DMOZ descriptions - so much for MSN being original!
3. They still cant work out the differance between an authority site and a blogg
4. The SERPS are still the worst on the internet and imo behind that of Gigablast and ASK Jeeves
I could go on.... They have a lot to do
What MSN needs to consider is some strategic partnerships with non microsoft browsers (and other things). For example I use opera and it's easy to search with google (just putting "g searchterm") but to search msn I first have to go to msn.com, wait for the page to load (which is a bit slow....needs to be quickened up for faster loading in my view), and then type the search in the search box.
As for crawling, MSN seems to be doing a better job as far as I can see than any other bots...at least on a medium sixed (90 pages so far) site that I'm building where msn bots are the most regular bot visitors.
Good that indexing is fast to, thus allowing users to get up to date info. For proof of this try a relatively new search time, for example "those machines are full".
465 results on yahoo
796 results on google
1,151 results on msn
I think MSN are definitely heading in the right direction. If they can keep scraper spam out (not that I've seen much) they're on to a winner.
One thing that would be nice by the way would be if the link colour would change on the SERPS for sites you've already visited.
But isn't this the SERPs before the update? At least when you type "search" you will find yahoo, google, msn there. But now, when you type "search", what would you get? Hmmm, kind of hard to believe that this update is heading to the right direction. Something is missing or way off in the algo.
The purpose of the site is completely non-commercial, so I can't justify spending the time to tweak it to suit MSN -- After all, it's been ranking fine since the days of AltaVista, and its purpose is to serve its visitors directly by providing information only. The result is that MSN users can no longer easily find the resources it offers, while users of other search services can.
The site still does fine for multiple-keyword searches, but it's historically been above the fold on page one for even the single-keyword search. Interestingly, it's the only site after the number one search result with an indented listing and a "Show more results from <this site>" link. All the other sites between the top result and ours on page three are single-page listings.
All sites in this special-interest sector are very small, so it may be a corner case, but after waiting several days to comment, it still looks like an authority issue to me. My overall impression is that results are better in some cases, worse in other cases, but inconsistent. That would be the concern, because an engineerable algorithm should be either better or worse, and not just better in some cases at the expense of other types of searches.
For some reason of late, I keep having to reload the MSN home page in order for anything to show up. Sometimes it takes three or four tries. I am using a Mac and Safari.
I know Mr. Gates is a PC/IE kinda guy ... but I think it would behoove MSN to at least try to accommodate those of us who aren't! ;)
(Or, I can stick with Google if you prefer?) :)