Forum Moderators: mack
The end result: For the 1 keyphrase that matters, the site did a major leap within a week across the board in Google, Yahoo, and MSN Beta, going from effectively off the radar, to top 40 in Google (and climbing so long as I do an update a day, which I haven't had the time to do), originally top 30 in Yahoo, but has recently climbed to top 20 (too soon to say if this is a stable climb in Yahoo).
Interestingly, the site went directly to the #1 spot in MSN Beta (not page 1, but the #1 spot, period), where it has remained ever since, aside from a 1 day drop on Boxing Day to the #3 spot.
Returned Search Results
Google: 20,700,000
Yahoo: 13,700,000
MSN Beta: 5,713,822
* At first, I thought the total # of responses was, at least in part, the reason for my higher placement in MSN. I no longer believe this is the case, however.
Keyphrase Competitiveness
High. Significant # of sponsor results for this keyphrase in all SE's, across the board.
Crawl Regularity
All three engines are crawling the site daily, with daily caches.
SERP Consistency
While the order is different for all three SEs, the top 50 results are essentially the same in all 3 SEs. The same sites are occupying the the list, just in different orders. Also, the top 10 is quite consistent (though the order differs somewhat) in all 3 SEs, my site being a notable anomaly in MSN. It is because the top 50 sites are so consistant across all three SEs, that I tend to discount the total number of responses for my high placement in MSN
So why the markedly different results?
This is, quite literally, keeping me up at nights. The #1 spot in both Google and Yahoo is held by a site that has been "online" (initially as a subscription based monthly e-mail) since 1991. If I mentioned the name of it, I can guarantee you would recognize it. To be honest, my site has no business knocking it out of the #1 spot. I have about 1% of the total amount of content available, and a very small fraction of the number of backlinks. I'm going to use this site as the comparison point, because of it's top placement in Google and Yahoo.
Backlinks
In theory, the lower number of total responses might well indicate that they're simply not seeing the backlinks. Using an extended backlink checking service, I have been able to determine that MSN registers nearly 5x the number of backlinks for the #2 site (we'll call it www.establishedcompetition.com from now on) as it does for the #1 site (www.mysite.com). Researching the quality of Backlinks, www.establishedcompetition.com has backlinks from more sites with higher PR as well. Because MSN is registering a higher number of backlinks for www.establishedcompetition.com , I am taking this as further evidence that my high placement has little to do with the smaller number of total responses returned in MSN search beta. It also would seem to indicate that MSN search Beta isn't placing a whole lot of weight on Backlinks.
DMOZ
Both www.mysite.com and www.establishedcompetition.com are registered in the same part of the directory of DMOZ, in which the search term I am using as a the focus of the analyses is a part of the DMOZ category. I can't draw any conclusions from this, as a result.
On Page Factors
This is where things start to diverge a bit. While both www.mysite.com and www.establishedcompetition.com have an almost identical number of occurances of the keyphrase on the home page, the similarity ends here. On www.mysite.com, the term has one specific occurance much closer to the name of the site in the body text. Also, www.establishedcompetition.com uses old style font tags and table structure, whereas www.mysite.com uses inline CSS, for fonts, but roughly the same type of tabling structure for layout. This indicates to me that both the placement of the search term, as well as the use of CSS, is taken into consideration by MSN Search Beta. Perhaps.
Unseen Code
Both www.establishedcompetition.com and www.mysite.com have remarkably similar Title attributes, with almost identical use of the keyowrds in the Title tag. Can't draw any conclusions from this, although I believe it plays a factor.
The major "unseen code" difference between the two pages is in the meta. www.establishedcompetition.com uses no meta tags whatsoever. www.mysite.com uses both the "description" and the "keywords" tags, with high prevalence placed on the chosen keyphrase. This is the single biggest difference between the two sites that weighs in www.mysite.com's favor.
Conclusions
I know I'm taking a big leap here, but I've come to the following conclusions:
MSN Search Beta, because it has a much newer database, doesn't currently give a rat's fart how old and well established a site is.
MSN Search Beta is also putting much less weight on Backlinks than the other engines. This could be for any number of reasons, including, perhaps, that someone at MSN has decided that backlinks have become an all too easy way of gaming the results in current SEs.
MSN seems to like CSS over font. This is well in line with other SEs. Keep placing your keyords in the H1 tags, and you'll keep MSN happy along with all the rest.
The Potentially Controversial Statement: MSN likes Meta, and puts a lot of weight on it. Without comparing it to various test pages, I have yet to determine if they use Meta as a standalone measure, or are using it only if it's in line with on page factors.
Opening up the Floor To Discussion
I could be totally wrong in all my assumptions, but if anyone here has any similar evidence, or evidence to the contrary, I wouldn't mind hearing about it.
Hello MSNdude:
Can you tell us anything about the importance of clean tight compliant (hopefully short) code?
Other things being equal, would a shorter page (5k-15k say) have any advantage over a much longer page?
- Larry
If MSN wanted to play favorites it would have been easy for them to bury a sites mine. I don't see that at all.
- msndude (msd)
Can you confirm that, to do well on MSN search, your website must be hosted in the country where your target market is.
Special I DONT like that many sites rank well just because they have a SINGLE page about the keyword searched for and the rest of the site is on a other topic, that realy messes up the results, there is absolute no theme ranking in there algo, I do rank well in MSN, but the results are nothing worth, this is also why we dont klnow how the ranking is working because the results are bad, no logic to it.
Another thing why is it that many sites arenot even include with 10% of there pages.
Its like.. you can't be in one AND the other without being hosted and having sites in each of the two geographic areas.
I have never seen this so absolute before although others have gone down the road to a lesser degree
I am also puzzled that (on .co.uk) they include a tick box for 'only from the UK' when that seem to be efectively the default.
Surely it would be better let the UK users (and other geographic areas) choose for themselves?
Cant wait for co.uk to go live. Nice job.
Meanwhile - i have to add my two pennys worth about "authority sites". Just because a site has featured well in Google because of a)age and b) links does not make it an authority. If the site is rich in content then yes, but many many sites in both Google and Yahoo that are listed high are not always rich in content. Some that are listed high due to links do not deserve to be.
Its unfair to assume that a high listed site in Google should also list high in MSN. After all what would be the point in using MSN if the results returned were the same as google.
Anyway, great job so far. Nice to get new player in the market
.co.uk version is already live at least here in Spain.
You may not be so chuffed when you see the results on the .co.uk version, unless of course your site is a .co.uk.
Oh, we're hosted in the Phillipines.
Many more of our pages that are "more geographically specific" (hint-hint) don't rank nearly as well for some reason.
Try a search from the box much lower down the page, not the top box.
I started getting visitors from this today, they are easy to spot as old serps are served by uk.search.msn.com
New serps are served by search.msn.co.uk
My traffic from msn is up a lot :) having jumped from nowhere to #1 (new site but probably deserves to be top page given the content)
Good luck
Quite pittyfull.
(1) sites that spam blogs rule the SERPS for a wide variety of phrases.
(2) I agree with several other comments that inbound anchor text doesn't seem to matter as much in MSN as in Google or Yahoo (which seems somewhat contradictory to #1 above). But perhaps the total number of links matters, while the exact anchor text is not as important. A single blog spammer will rank very highly for a broad set of terms.
(3) this sounds illogical, but it appears that close to half of the results are random. obviously I haven't figured out the algo, but I can find NO reason that certain sites are #1, or in the top 10. very poor quality, semi-relevant sites/pages that don't deserve to be in the top 500 are filling half of the top slots. I realize that MSN's algo is very different from the others, but when analyzing these "random" sites, I can find nothing (on or off page) that justifies ranking so high.
half of the results are random
It makes complete sense to serve up a random sample, and I'm surprised other SEs don't do it. Here's why:
The random sample can be used to automatically tune the algorithm through machine learning by using a feedback metric (clickthrough rate, for example).
Pages with higher score rise to the top. This assures the SE shows the "most relevant/popular/whatever" pages over time.
It also allows the SE to correlate certain page properties with metric score to tweak the algo down the road.
A site of mine only had 6% of pages in the beta index until earlier today when it jumped to 100%, being a recently developed site I am able to see the reasons behind this and the crawling pattern.
1. The full crawl was done on Jan 12th - so it took about a week to filter through.
2. The first page requested (homepage) was definitely the result of the bot following a link from a page that is ranked well in the old MSN - I know this as they put identifiers on their outbound links.
3. It ate all of the pages in one sitting, as it had never visited pages that were linking to other pages crawled in that sitting it MUST crawl some links immediately rather than report them back for crawling at a later date.
4. Results from the new pages added today. Good, but there is still a preference for the homepage. e.g. there is a page which is 2 clicks from the homepage, it talks about one specific area of the service provided. For the 2 word phrase (title & heading) the page ranks worse than the homepage which has the 2 words seperately in the body by chance. SO either the weighting of the links to the homepage are making this happen OR MSN thinks the homepage is close enough and sends you there.
5. The main MSN.co.uk results are now testing the beta data, it had been restricted to less prominent searchboxes.
Hope this helps, any comments?
It's a shame that it's not a 'money' term. It's a service that is well searched though.
Anyone else getting a similar result to try and spot a pattern?
One possibilty is that MSN ranks logical AND combinations highly, in our case it would be
keywords in:
anchor text AND page title AND <hx>headings AND filename
if you are missing any one of these, the term goes to zero...just thinking out loud here.
C
It makes complete sense to serve up a random sample, and I'm surprised other SEs don't do it.
Your logic is really very interesting for me - particularly on less common / less competitive searches. it is not an idea that I have seen before. i don't think MSN are doing it if you want my tuppence worth, but I do think there are instances where it would help to define the algorithm. Sadly, if noticed, it might also be open to abuse as well.
C
It would only be useful where the algo cannot finf other high likely relevency I would guess.
I don't have any of my keywords in H[x] Header tags, and neither did two of my competitors. All of us are buried in the listings. My competitors started using header tags (I keep track of my competition very closely), and they moved to the first page. I have done the same thing, and im waiting for the next crawl.
It appears that for MSN you must meet all the basic criteria to be ranked well. No matter how much seo you do, if you miss one item like header tags, you are dead in the water.
For example, when I search for my entire keyword specific title in the search engines, Im number one, but on MSN, im burried on page 18. All the sites that link to me, which has H1 headers, are listed before me.
Im not very impressed. And if this is MSN beta of the new search engine, it should be buried deep, and not the primary method (and only method) of searching on MSN.
Its time for MSN to bring 'Coke Classic' back, until at least their beta release is up to par.
DirectHit became Teoma. And for our industry, I find this about the most spam free search engine. I know my competiton, and they are all there on the frontpage, with no 'wanabee's'.
Its to bad that teoma never caught on. Here is another good example of good product-bad marketing syndrome.
MSN requires "anchor text AND page title AND <hx>headings AND filename"-- the data I see supports that theory.
New msn is good old pre-florida google that we all loved to bits! At least in many many ways. Old good SEO still works - non-spammy anchors (without that bloody apparent class C or whatever-else-similar-IP filtering by google), h elements, descriptive metas, 7-10% onpage keyword density. Big sites with repeated semantic core throught the site are valued higher as I have noticed. And you know what - they ARE bloody RELEVANT!
At least 80% is anchors though. I ranked my brandnew site in top5 for some very serious floral searches in 2 months just by doing VERY aggessive link building. No spam, very few anchors repeated, max. 4 keywords - worked like hell for me. Must have got me to the life-long sandbox imprisonment though...
But you know what - I couldnt care less now. Being #1 for 2 keyword money phrase in msn now is more or less the same as being #7 or 8 in google now - but the ways or achieving it are not nearly as perverted!
Google if you think I and my business are not as reputable as big mister oldie on top of your SERPs - i am fine with that.
I know that one day Billie integrates msn into the new Office or gives msn search a wider coverage in longhorn by any other ways - just take your time. This will be the end of sandbox worries for most of us.
You hear that greedy google monsters?
