homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Microsoft / Bing Search Engine News
Forum Library, Charter, Moderators: mack

Bing Search Engine News Forum

Bing/MSN needs to update the HTTP ACCEPT header.
If someone from Microsoft is around, this is for you!
JAB Creations

WebmasterWorld Senior Member jab_creations us a WebmasterWorld Top Contributor of All Time 10+ Year Member

Msg#: 4111388 posted 2:04 pm on Apr 7, 2010 (gmt 0)

I recently updated the initial version of a basic statistics script to start recording various headers. I firmly believe in serving the same content to humans and spiders however since my site is XHTML (served correctly as application/xhtml+xml) there are a lot of issues I have to deal with; mainly IE's lack of support (IE9 now supports XHTML though) and the questionable support among legitimate search spiders.

The main obstacle to serving pages universally as XHTML (once IE8 and older market shares are beyond even business's cryptic desire for support) is the lack of HTTP ACCEPT header being set or too vague. Currently the only major search spider (at least here in the United States over the past few days) to correctly serve the HTTP ACCEPT header is Yahoo. Most spiders that do have the HTTP ACCEPT header set only declare support for "*/*" which is completely unusable as there are many instances of Internet Explorer serving this as the entire HTTP ACCEPT string. For XHTML sites to correctly serve XHTML as application/xhtml+xml it is absolutely necessary to detect the string "application/xhtml+xml" instead of the HTTP ACCEPT header.

I hope that someone over at Microsoft could forward this information to the folks running MSNbot so if it does understand XHTML as application/xhtml+xml then they could set the HTTP ACCEPT header to include the string. Here is an example of a browser's HTTP ACCEPT header...


Remember, it is absolutely necessary to see the string application/xhtml+xml in the header otherwise even if */* is present the server has to presume the user agent only understands text/html due to Internet Explorer using only that string.

- John


Global Options:
 top home search open messages active posts  

Home / Forums Index / Microsoft / Bing Search Engine News
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved