Msg#: 3905573 posted 5:21 pm on May 2, 2009 (gmt 0)
Here are response headers from both search.live.com and search.msn.com. Notice that Connection header has an extrenious "Transfer-Encoding"
HTTP/1.1 200 OK Content-Type: text/html; charset=utf-8 X-Powered-By: ASP.NET P3P: CP="NON UNI COM NAV STA LOC CURa DEVa PSAa PSDa OUR IND", policyref="http://privacy.msn.com/w3c/p3p.xml" Vary: Accept-Encoding Content-Encoding: gzip Date: Sat, 02 May 2009 16:56:37 GMT Transfer-Encoding: chunked Connection: keep-alive, Transfer-Encoding Cache-Control: private
I've noticed this problem today using Firefox browser with Live HTTP headers extension. Earlier i reported bad header issues with msnbot. That header error (typo?) seems not to have been fixed yet. Hey MSNDude what's up with MSN and the http header problems?
Msg#: 3905573 posted 5:34 pm on May 2, 2009 (gmt 0)
They've got other HTTP header problems as well. I note that a particular variant of msnbot/1.1 sends an HTTP From: header reading: From: If-Modified-Since <appropriate date here> when fetching pages.
Apparently, they don't have much in the way of quality assurance checking at the most fundamental protocol levels. I'm not sure how you can build up a good search engine when the critical infrastructure has bugs... :(