Yahoo does not honor Content-Type header

Yahoo relies on file extension not in content-type

[tecsisa]

I just found that Yahoo doesn't use the Content-Type header when a file extension that it doesn't know appear.

We have two sites, one uses .php as extension and other uses .igw (iGesWeb) as extension.

Since the iGesWeb site is build with a java CMS we used a not known extension to filter those requests an produce the page.

Reviewing logs we see that Yahoo visits the iGesWeb site frecuently but only 2 pages are indexed. Why?

After many posts in the site explorer a Yahoo employee suggests:

"The other two sitemaps are running into problems due to the '.igw' extension. Can you configure your CMS to output to one of the standard file extensions? "

I totaly dissagree with that! Yahoo should honor Content-Type header.

It says "text/html;charset=UTF-8" and that should be enough. Why do I have to use a "standard" extension if no standar extension represents the kind of processing our CMS does?

We have no problems with any other search engines and changing extension will affect atleast 5.000 pages.

[edited by: encyclo at 7:29 pm (utc) on June 24, 2007]
[edit reason] no links to personal sites please [/edit]