MSN UK - doesn't like new TLDs

... avoid for UK based customers

[4eyes]

I never had to worry about this before, but a recent episode with a new customer brought it to my attention.

.info
.biz
.tv

Are all being filtered by MSN UK.

Sites which rank well on MSN.com are being penalised on MSN.co.uk, (appearing after all the other results)

I guess that the other new TLDs are also being penalised as well - can't find enough of them to test for sure.

[jmccormac]

Not sure if this is a case of MSN.co.uk adopting the quick fix solution for SEs to get into national (cctld) markets. Most SEs (eg Google) seem to look at the cctld as being a fast way to rapidly build an index of related websites. With a large cctld like .uk this may work but on a small cctld like .ie (where from stats, I think that only 13K or so active websites exist) the results can be pretty dire. The last time I spidered .info , I think that there were about 375K websites in the search. Many of these were holding pages/swamps. I have not looked at .biz or .tv yet.

Using a ranking based algorithm on a relatively new tld such as .info or .biz is a bad thing. Ranking algorithms are best for large sets where there is a lot of relations. With .info and .biz, such links would be far fewer in number. Active websites (as opposed to holding pages or swamps (the future home of * site)) are more visible in smaller tlds like .info. One such swamp had about 35K websites pointing to it. Another side-effect of a ranking algorithm approach would be that the number of .info/.biz/.tv domains referred to by .com/.net/.org (CNO) or .uk domains would be small and perhaps well short of the total number of websites in a tld.

If MSN uses a ranking algorithm approach, then the knock-on effect would be to downgrade rather than filter the results from the newer tlds but at the same time boost the cctld results to give it a local flavour.

Regards...jmcc

[1] August 2002 - It took about a week to check that websites existed and then about five days to spider the index pages on a 128K leased line.

[1][edited by: engine at 7:15 am (utc) on Sep. 24, 2002]
[edit reason] no sigs, please. [/edit]

[4eyes]

jmmc

There may be more to it, but I suspect that this is a much simpler issue, such as Inktomi feeding MSN a UK dataset which treats .info etc as being a non-uk domain.

I see the same problem with Espotting's Inktomi results.

Whatever the reason, .info and its brothers are not suitable for sites targeting the UK:(

[GetVisible]

perhaps they look at the location of sites that link to it and the sites that it links to?

say 80% of the sites that link to the *.info and 80% of the sites it links to are uk then it will class the *.info site as located in the uk?

[jmccormac]

Yep 4eyes,
I think that the non-localisation of the data does cause problems like this. Also with .uk the number of websites is large (I think only Germany and Spain approach it in size). The upside of having a large number of websites in the .uk is that many will be on identifiably UK IP ranges so it would allow them to optimise the dataset.
However without actually seeing how they did it, it is difficult to say. Given the results from MSN that I've seen on WhoisIreland, I think that no IP based filtering is being used.

Regards...jmcc

[jmccormac]

In msg #4 GetVisible posted:
"perhaps they look at the location of sites that link to it and the sites that it links to?
say 80% of the sites that link to the *.info and 80% of the sites it links to are uk then it will class the *.info site as located in the uk? "

This is the weakness of an algorithm based on interrelatedness GetVisible. You would have to know that the sites you are comparing against are actually UK sites. There are two basic ways of approaching the problem:
1: Use the Ranking Approach based on sites that link to sites.
2: A Logically Optimised approach.

Google's main strength is its ranking algorithm. Without a lot of interlinked sites, a ranking algorithm will not work well and you will get spurious results.

A logically optimised approach would use begin with the complete set of websites and start to prune the list down using indicators such as web content, language, IP ranges etc. It is far more effective for a national SE. (Though in the case of Irish owned .info, the first trawl based on these techniques only detected 129 Irish owned .info domains.) I've used similar techniques to identify Irish owned .com/.net/.org websites. The same techniques would work for any European country's cctld.

The .info tld is still a very small domain about the size of a larger European cctld. The .uk is much bigger. From a marketing point of view, people still think in .com terms. Many .info sites forward to .com or, more rarely, cctld websites. However the big problem for SEs is in determining the number of active websites in a particular cctld. If SEs applied a 'Last Modified' approach, then a very high percentage of websites would appear as ghost sites. The effect is fairly clear with .ie - of websites returning 'Last-Modifed' info:

Websites Last Updated 2002: 7847
Websites Last Updated 2001: 3341
Websites Last Updated 2000: 1239

Now the .ie is poorly managed and the deletion of lame and dead domains has not been happening on a regular basis for the last few years. This would mean that the domain count is artifically high. This is the same kind of thing that has been seen with .com/.net/.org. Though millions of speculative domains have been deleted in the last year. The high price of .ie domains also contributed to its low takeup. With the .uk, the price is far lower and consequently the take up has been higher, and naturally speculative and non-developed domains abound.

It would be better to concentrate any marketing on .uk sites for the UK as it seems that these larger SEs are using the quick and dirty method of generating an index. Of course an optimised UK only SE based on a logically optimised approach would beat the bigger SEs but their advertising budgets are bigger. :)

Regards...jmcc