| This 63 message thread spans 3 pages: < < 63 ( 1  3 ) > > || |
|Analyzing why I'm doing well in MSN Search|
Thinking out loud, feel free to correct my assumptions.
| 11:27 pm on Jan 6, 2005 (gmt 0)|
Background: At the end of November, I launched a major makeover of a site that has been around for a few years, and the makeover was specifically targetting SEO factors, using "on page" factors, almost exclusively "white hat" techniques. The techniques used were ones I gleaned primarily from WebmasterWorld, with a few SEO blogs as secondary resources.
The end result: For the 1 keyphrase that matters, the site did a major leap within a week across the board in Google, Yahoo, and MSN Beta, going from effectively off the radar, to top 40 in Google (and climbing so long as I do an update a day, which I haven't had the time to do), originally top 30 in Yahoo, but has recently climbed to top 20 (too soon to say if this is a stable climb in Yahoo).
Interestingly, the site went directly to the #1 spot in MSN Beta (not page 1, but the #1 spot, period), where it has remained ever since, aside from a 1 day drop on Boxing Day to the #3 spot.
Returned Search Results
MSN Beta: 5,713,822
* At first, I thought the total # of responses was, at least in part, the reason for my higher placement in MSN. I no longer believe this is the case, however.
High. Significant # of sponsor results for this keyphrase in all SE's, across the board.
All three engines are crawling the site daily, with daily caches.
While the order is different for all three SEs, the top 50 results are essentially the same in all 3 SEs. The same sites are occupying the the list, just in different orders. Also, the top 10 is quite consistent (though the order differs somewhat) in all 3 SEs, my site being a notable anomaly in MSN. It is because the top 50 sites are so consistant across all three SEs, that I tend to discount the total number of responses for my high placement in MSN
So why the markedly different results?
This is, quite literally, keeping me up at nights. The #1 spot in both Google and Yahoo is held by a site that has been "online" (initially as a subscription based monthly e-mail) since 1991. If I mentioned the name of it, I can guarantee you would recognize it. To be honest, my site has no business knocking it out of the #1 spot. I have about 1% of the total amount of content available, and a very small fraction of the number of backlinks. I'm going to use this site as the comparison point, because of it's top placement in Google and Yahoo.
In theory, the lower number of total responses might well indicate that they're simply not seeing the backlinks. Using an extended backlink checking service, I have been able to determine that MSN registers nearly 5x the number of backlinks for the #2 site (we'll call it www.establishedcompetition.com from now on) as it does for the #1 site (www.mysite.com). Researching the quality of Backlinks, www.establishedcompetition.com has backlinks from more sites with higher PR as well. Because MSN is registering a higher number of backlinks for www.establishedcompetition.com , I am taking this as further evidence that my high placement has little to do with the smaller number of total responses returned in MSN search beta. It also would seem to indicate that MSN search Beta isn't placing a whole lot of weight on Backlinks.
Both www.mysite.com and www.establishedcompetition.com are registered in the same part of the directory of DMOZ, in which the search term I am using as a the focus of the analyses is a part of the DMOZ category. I can't draw any conclusions from this, as a result.
On Page Factors
This is where things start to diverge a bit. While both www.mysite.com and www.establishedcompetition.com have an almost identical number of occurances of the keyphrase on the home page, the similarity ends here. On www.mysite.com, the term has one specific occurance much closer to the name of the site in the body text. Also, www.establishedcompetition.com uses old style font tags and table structure, whereas www.mysite.com uses inline CSS, for fonts, but roughly the same type of tabling structure for layout. This indicates to me that both the placement of the search term, as well as the use of CSS, is taken into consideration by MSN Search Beta. Perhaps.
Both www.establishedcompetition.com and www.mysite.com have remarkably similar Title attributes, with almost identical use of the keyowrds in the Title tag. Can't draw any conclusions from this, although I believe it plays a factor.
The major "unseen code" difference between the two pages is in the meta. www.establishedcompetition.com uses no meta tags whatsoever. www.mysite.com uses both the "description" and the "keywords" tags, with high prevalence placed on the chosen keyphrase. This is the single biggest difference between the two sites that weighs in www.mysite.com's favor.
I know I'm taking a big leap here, but I've come to the following conclusions:
MSN Search Beta, because it has a much newer database, doesn't currently give a rat's fart how old and well established a site is.
MSN Search Beta is also putting much less weight on Backlinks than the other engines. This could be for any number of reasons, including, perhaps, that someone at MSN has decided that backlinks have become an all too easy way of gaming the results in current SEs.
MSN seems to like CSS over font. This is well in line with other SEs. Keep placing your keyords in the H1 tags, and you'll keep MSN happy along with all the rest.
The Potentially Controversial Statement: MSN likes Meta, and puts a lot of weight on it. Without comparing it to various test pages, I have yet to determine if they use Meta as a standalone measure, or are using it only if it's in line with on page factors.
Opening up the Floor To Discussion
I could be totally wrong in all my assumptions, but if anyone here has any similar evidence, or evidence to the contrary, I wouldn't mind hearing about it.
| 10:25 am on Jan 18, 2005 (gmt 0)|
Hi Inbound: regarding:
" 1. All pages on the site are AAA compliant. "
.. is that something like being W3C compliant, or does it have more to do with auto towing insurance?
Can you tell us anything about the importance of clean tight compliant (hopefully short) code?
Other things being equal, would a shorter page (5k-15k say) have any advantage over a much longer page?
| 2:56 pm on Jan 18, 2005 (gmt 0)|
Regarding favortism, my pages are ranking very well in MSN (and only in MSN) despite touting all sorts of non-MS technologies in meta and elsewhere.
If MSN wanted to play favorites it would have been easy for them to bury a sites mine. I don't see that at all.
| 6:04 pm on Jan 18, 2005 (gmt 0)|
Hi Larryhatch. We cannot comment specifically on how ranking considers page size. A couple of related thoughts are 1.) overall page size should be kept under 150KB of html and 2.) cross browser compliant code is good for your users. Both of these are also user benefits. Users on narrowband will be happy that they do not have to download massive pages and compliant code will ensure that your site works well in all browsers.
- msndude (msd)
| 7:39 pm on Jan 18, 2005 (gmt 0)|
Thank You MsnDude. It appears that sites that are designed for the US market, but are hosted outside the US (eg Canada) are penalized, in your rankings.
Can you confirm that, to do well on MSN search, your website must be hosted in the country where your target market is.
| 12:51 am on Jan 19, 2005 (gmt 0)|
Not sure about that regional bias. My stuff is all on a canadian hosting service, and I'm not seeing any penalties at all.
| 1:51 am on Jan 19, 2005 (gmt 0)|
Just wanna comment on the quality of the referrals from MSN beta. I have a small ecommerce site written from ground up using .NET. The site provides detailed statistics of visitors activity including landing pages, visited pages, time spent, etc. Based on my observation for the past two days MSN referrals are of very high quality – people stick around instead of pushing the back button. Obviously my site is relevant for the search terms used. It is a very positive development for me and a nice change from google’s scraps. I just want to complement MSN for the job well done and hope they figure out a way to keep their index spam free. Long Live MSN!
| 12:41 pm on Jan 19, 2005 (gmt 0)|
I think we have had a yes and now for every SEO rules, this is once again sign for that the results are very hard to get wise on.
Special I DONT like that many sites rank well just because they have a SINGLE page about the keyword searched for and the rest of the site is on a other topic, that realy messes up the results, there is absolute no theme ranking in there algo, I do rank well in MSN, but the results are nothing worth, this is also why we dont klnow how the ranking is working because the results are bad, no logic to it.
Another thing why is it that many sites arenot even include with 10% of there pages.
| 1:23 pm on Jan 19, 2005 (gmt 0)|
Mmm I know this has been pointed out before but I see two complete different data sets when changing from .com to .co.uk
Its like.. you can't be in one AND the other without being hosted and having sites in each of the two geographic areas.
I have never seen this so absolute before although others have gone down the road to a lesser degree
I am also puzzled that (on .co.uk) they include a tick box for 'only from the UK' when that seem to be efectively the default.
Surely it would be better let the UK users (and other geographic areas) choose for themselves?
| 1:40 pm on Jan 19, 2005 (gmt 0)|
The msn search is superb. Less spam than both google and Yahoo and for once is a search engine that is focused firmly on content which i feel google and yahoo have let us down on over the last couple of years.
Cant wait for co.uk to go live. Nice job.
Meanwhile - i have to add my two pennys worth about "authority sites". Just because a site has featured well in Google because of a)age and b) links does not make it an authority. If the site is rich in content then yes, but many many sites in both Google and Yahoo that are listed high are not always rich in content. Some that are listed high due to links do not deserve to be.
Its unfair to assume that a high listed site in Google should also list high in MSN. After all what would be the point in using MSN if the results returned were the same as google.
Anyway, great job so far. Nice to get new player in the market
| 2:05 pm on Jan 19, 2005 (gmt 0)|
Yes, yes RichT nice job, good postions, bring on the dosh etc etc...but please read again my post above.
.co.uk version is already live at least here in Spain.
You may not be so chuffed when you see the results on the .co.uk version, unless of course your site is a .co.uk.
| 2:08 pm on Jan 19, 2005 (gmt 0)|
regarding positioning well in beta:
we have 50 U.S.-geographic-related pages (you can probably guess what those 50 keywords are) that are #1 (or close to it) in beta for keyword pairs, but nowhere near that in the other SE's. The factors that might be at work appear to be:
-keyword pairs in title
-keyword pairs in page name
-1 keyword in domain name
- keyword pairs in h3
- numerous links with keyword pairs from pages on another, related-theme domain of ours
Oh, we're hosted in the Phillipines.
Many more of our pages that are "more geographically specific" (hint-hint) don't rank nearly as well for some reason.
| 2:14 pm on Jan 19, 2005 (gmt 0)|
MSN.co.uk has started live testing of the beta results
Try a search from the box much lower down the page, not the top box.
I started getting visitors from this today, they are easy to spot as old serps are served by uk.search.msn.com
New serps are served by search.msn.co.uk
My traffic from msn is up a lot :) having jumped from nowhere to #1 (new site but probably deserves to be top page given the content)
| 8:24 pm on Jan 19, 2005 (gmt 0)|
On some of my key keywords I'm doing ok, but it will be interesting to see how that translates into traffic, one phrase is technical, very technical, I expect to see almost no increase in traffic from that from msn, one is very general, I expect to see big traffic boost from that, and one is a good money term, I'll have to keep a close watch on msn and see how it does for these sites.
| 10:48 pm on Jan 19, 2005 (gmt 0)|
The top site for our keyword has a zero second meta refresh. The page only shows for a very short time, showing tons of spam.
| 11:10 pm on Jan 19, 2005 (gmt 0)|
I have many reasons to believe that the results we see at the moment have nothing to do with the results after a month or so,msn bot has not yet indexed the 90% of billions of sites, consencuently minus 90% of inbound links from those millions of sites. The results you see at the moment are from pages that randomly and accidentally have been more then 60% indexed plus there inbound links, (thats probably a reason that you see many enthusiastic members to celebrate the top 1 position for there new sites that never had score before in G or yahoo). When the crawl will finish then I believe you will see the stable results. That's my theory anyway.
| 11:22 pm on Jan 19, 2005 (gmt 0)|
some observations in a couple of areas I follow:
(1) sites that spam blogs rule the SERPS for a wide variety of phrases.
(2) I agree with several other comments that inbound anchor text doesn't seem to matter as much in MSN as in Google or Yahoo (which seems somewhat contradictory to #1 above). But perhaps the total number of links matters, while the exact anchor text is not as important. A single blog spammer will rank very highly for a broad set of terms.
(3) this sounds illogical, but it appears that close to half of the results are random. obviously I haven't figured out the algo, but I can find NO reason that certain sites are #1, or in the top 10. very poor quality, semi-relevant sites/pages that don't deserve to be in the top 500 are filling half of the top slots. I realize that MSN's algo is very different from the others, but when analyzing these "random" sites, I can find nothing (on or off page) that justifies ranking so high.
| 3:58 pm on Jan 20, 2005 (gmt 0)|
|half of the results are random |
It makes complete sense to serve up a random sample, and I'm surprised other SEs don't do it. Here's why:
The random sample can be used to automatically tune the algorithm through machine learning by using a feedback metric (clickthrough rate, for example).
Pages with higher score rise to the top. This assures the SE shows the "most relevant/popular/whatever" pages over time.
It also allows the SE to correlate certain page properties with metric score to tweak the algo down the road.
| 5:45 pm on Jan 20, 2005 (gmt 0)|
More info on MSN
A site of mine only had 6% of pages in the beta index until earlier today when it jumped to 100%, being a recently developed site I am able to see the reasons behind this and the crawling pattern.
1. The full crawl was done on Jan 12th - so it took about a week to filter through.
2. The first page requested (homepage) was definitely the result of the bot following a link from a page that is ranked well in the old MSN - I know this as they put identifiers on their outbound links.
3. It ate all of the pages in one sitting, as it had never visited pages that were linking to other pages crawled in that sitting it MUST crawl some links immediately rather than report them back for crawling at a later date.
4. Results from the new pages added today. Good, but there is still a preference for the homepage. e.g. there is a page which is 2 clicks from the homepage, it talks about one specific area of the service provided. For the 2 word phrase (title & heading) the page ranks worse than the homepage which has the 2 words seperately in the body by chance. SO either the weighting of the links to the homepage are making this happen OR MSN thinks the homepage is close enough and sends you there.
5. The main MSN.co.uk results are now testing the beta data, it had been restricted to less prominent searchboxes.
Hope this helps, any comments?
| 5:57 pm on Jan 20, 2005 (gmt 0)|
P.S. had a test running on popular terms for my site, more than 90% are on the first page! That is nuts, I think many are justified but I've never seen such a broad range of terms ranking well for any of my sites.
It's a shame that it's not a 'money' term. It's a service that is well searched though.
Anyone else getting a similar result to try and spot a pattern?
| 1:30 pm on Jan 24, 2005 (gmt 0)|
others report that anchor text doesn't seem to matter much. yet this is the most unique feature of our pages that rank at or close to #1 in MSN search, but nowhere near that in other SE's.
One possibilty is that MSN ranks logical AND combinations highly, in our case it would be
anchor text AND page title AND <hx>headings AND filename
if you are missing any one of these, the term goes to zero...just thinking out loud here.
| 8:38 pm on Jan 24, 2005 (gmt 0)|
All the sites occupying the top 10 on keywords I am checking utilize Header tags.
| 9:22 pm on Jan 24, 2005 (gmt 0)|
|It makes complete sense to serve up a random sample, and I'm surprised other SEs don't do it. |
Your logic is really very interesting for me - particularly on less common / less competitive searches. it is not an idea that I have seen before. i don't think MSN are doing it if you want my tuppence worth, but I do think there are instances where it would help to define the algorithm. Sadly, if noticed, it might also be open to abuse as well.
| 4:22 am on Jan 25, 2005 (gmt 0)|
I can't see why search engines would use click-through data to determine relevancy. Maybe I am missing something, but IMHO higher click throughs simply mean more attractive titles--not more relevant content. Now, if there was a way to track the visitors movement throughout the website (via toolbar, etc) that might be a better indicator of relevance (more than one page view, etc). But to say that a site that has higher click throughs from the serps implies relevancy is short sighted.
| 5:34 am on Jan 25, 2005 (gmt 0)|
Maybe, although Direct hit used to do this as I recall.
It would only be useful where the algo cannot finf other high likely relevency I would guess.
| 1:26 pm on Jan 26, 2005 (gmt 0)|
Well if you don't use one of MSM's key criteria, you are dead in the water.
I don't have any of my keywords in H[x] Header tags, and neither did two of my competitors. All of us are buried in the listings. My competitors started using header tags (I keep track of my competition very closely), and they moved to the first page. I have done the same thing, and im waiting for the next crawl.
It appears that for MSN you must meet all the basic criteria to be ranked well. No matter how much seo you do, if you miss one item like header tags, you are dead in the water.
For example, when I search for my entire keyword specific title in the search engines, Im number one, but on MSN, im burried on page 18. All the sites that link to me, which has H1 headers, are listed before me.
Im not very impressed. And if this is MSN beta of the new search engine, it should be buried deep, and not the primary method (and only method) of searching on MSN.
Its time for MSN to bring 'Coke Classic' back, until at least their beta release is up to par.
| 4:56 pm on Jan 26, 2005 (gmt 0)|
Lots of movement in the new MSN over the past week, (at least for our areas). Only trouble is we started out very strong with a variety of sites, however a fair chunk of them are just sort of slowly sinking down. No radical drops or anything, just a slow movement the wrong way.
| 7:29 pm on Jan 26, 2005 (gmt 0)|
I imagine that those who were near the top will find new competition, as all the spammers realize how easy it is to do well in MSN.
DirectHit became Teoma. And for our industry, I find this about the most spam free search engine. I know my competiton, and they are all there on the frontpage, with no 'wanabee's'.
Its to bad that teoma never caught on. Here is another good example of good product-bad marketing syndrome.
| 2:00 pm on Jan 27, 2005 (gmt 0)|
-- the data I see supports that theory.
|MSN requires "anchor text AND page title AND <hx>headings AND filename" |
| 4:16 pm on Jan 27, 2005 (gmt 0)|
If MSN requires the filename, then the home page should not rank that well. My home page is most optimized, and since 'index' and 'default' are not
my keywords, this would explain why my secondary pages are ranking better than my optimized pages.
| 5:23 pm on Jan 27, 2005 (gmt 0)|
C'mon guys - good days are back!
New msn is good old pre-florida google that we all loved to bits! At least in many many ways. Old good SEO still works - non-spammy anchors (without that bloody apparent class C or whatever-else-similar-IP filtering by google), h elements, descriptive metas, 7-10% onpage keyword density. Big sites with repeated semantic core throught the site are valued higher as I have noticed. And you know what - they ARE bloody RELEVANT!
At least 80% is anchors though. I ranked my brandnew site in top5 for some very serious floral searches in 2 months just by doing VERY aggessive link building. No spam, very few anchors repeated, max. 4 keywords - worked like hell for me. Must have got me to the life-long sandbox imprisonment though...
But you know what - I couldnt care less now. Being #1 for 2 keyword money phrase in msn now is more or less the same as being #7 or 8 in google now - but the ways or achieving it are not nearly as perverted!
Google if you think I and my business are not as reputable as big mister oldie on top of your SERPs - i am fine with that.
I know that one day Billie integrates msn into the new Office or gives msn search a wider coverage in longhorn by any other ways - just take your time. This will be the end of sandbox worries for most of us.
You hear that greedy google monsters?
| 8:31 pm on Jan 27, 2005 (gmt 0)|
exact phrase match also seems to count more than usual..maybe...
| This 63 message thread spans 3 pages: < < 63 ( 1  3 ) > > |