Unicode in FireFox

[kembreg]

Saw the oddest thing today:

Check out this site in both IE and FireFox:


www.linguashop.com/fr


In IE the accents such as é, â etc look fine.
In Firefox, all you see are ? marks.

The header is set to utf-8.

Any idea why Firefox is not showing the text properly?

Any feedback would be appreciated.

David

[tedster]

There is a bit of a mystery here - I do see UTF-8 is declared both in the HTTP header and in the HTML charset meta tag. The problem also appears in Safari and Chrome, but not in Opera.

However, when I View Source, it looks like some other character is being used in the text, and not the actual Unicode é, â or whatever.

[claus]

Firefox will generally assume that if you have declared UTF-8 as "charset", then you know what you are doing, and the document should be interpreted as UTF-8.

IE, OTOH, generally assumes that you don't know what you're doing. So, if IE is able to interpret some special characters it will do so, even when the doctype tells it otherwise.

Firefox is right, and Firefox *is* showing the text properly. You should trust the document author.

So, what is happening? Well, this happens frequently with static html-files, if there's a conflict between what the server thinks the document is, what the charset meta tag says it is, and what the raw html file was saved as before uploading it to the server. It may happen with asp or php or any other type of file as well.

As an example, your server may think that all your files are UTF-8 because that is in the config file, and you may even have written so in your charset meta tag for the document itself... but your text editor has saved your document as ISO-8815-2 or ANSI, and that is what you have uploaded.

That's when you get those small questionmarks.

:)

---
PS: I don't think the TOS of this board allows you to link to your site.

[tedster]

We're going to "bend" the rules about pointing people to your own website for this one time - it seems to me that the diagnosis would be more difficult otherwise.

I'm assuming that the text you have is pasted in from a word processing program of some kind, right?

[claus]

One extra thought I forgot about before.

If your pages are database-generated it may be the case that the database (or the individual table) stores its content in another character set than UTF-8. For the MySQL database you would look for the property called "collation".

[encyclo]

Welcome to WebmasterWorld kembreg :)

When I tested the page, the accents didn't display correctly in IE8 either. The explanation is simple - the text is not actually encoded in UTF-8, despite the charset declaration. In fact, the page is currently encoded in ISO-8859-1. (Or ISO-8859-15 or windows-1252, which are functional equivalents of ISO-8859-1 apart from some minor differences)

For a temporary fix for you only, go to the View menu in Firefox and choose ISO-8859-1 from the list, and the accents will display correctly.

The permanent fix is either to declare the current charset instead of UTF-8, or to convert the text to UTF-8 encoding.

[kembreg]

Thanks all!
You're right.
I don't know why I didn't think of it.
PHP is getting the text from resource files which are not (yet) stored as UTF-8.

I'll have to switch the resource files to UTF-8 and see how it goes.

David

[edited by: tedster at 6:38 am (utc) on Jul 9, 2010]

[commanderW]

It looks like this particular problem has been solved. I would like to add here that browsers do not have universal or identical support for Unicode. Neither do operating systems.

I would like to point to this UTF-8 browser test page [fileformat.info ]

and to the Allen Wood Unicode site [alanwood.net ]. Parts of it are a bit out of date, but there are many links to the details of Unicode and he focuses on the little known problem of Unicode browser compatibility.

Safari, Firefox and Opera still do not have complete Unicode support. Somehow, this is independant of how good the support or installation in your OS is. In my experience it is generally confined to obscure areas like the Miscellaneous Symbols block. [fileformat.info ]

It is worth checking for this difference in support from time to time. I mention this in case you may have 2 problems in this area. Non-UTF-8 encoded text files, and possibly, because of the comments on browser differences in display, this other problem .

[phranque]

this current thread in webmaster general addresses many of these issues:
struggling with chinese characters, in MySQL/PHP [webmasterworld.com]

perhaps there are some interesting insights for you there.