Welcome to WebmasterWorld Guest from 18.104.22.168
Forum Moderators: martinibuster
So, despite a 301 redirect from example.com to www.exmaple.com that actually predates the DNS going live ten years ago, Slurp has "inferred" this non-www domain and is now showing it and linking to it in their search results.
Thanks for the referrals, Yahoo -- But I sure wish that every one of them didn't result in a redirect because the domain you're linking to is incorrect!
If anyone else sees something similar, don't panic -- This is definitely something wrong on their end -- Either now or in the recent past (the Yahoo cache date on the page I looked at was Nov 19th, 2008).
I'm just amazed that Yahoo and MSN Live continue to tolerate flaws in their fundamental infrastructure that sabotage any chances they have of competing with Google -- It amazes me that they think they can produce great search results when their crawlers evidently do not understand and/or properly handle basic HTTP signaling... They are apparently oblivious that they're building their houses on swampy, unstable ground. :o
In this case, I believe the cause may be related to Yahoo's very-annoying habit of "inventing" and making requests for URLs even when there are no links on the Web pointing to those resources. An example is the directory-level URLs that Slurp attempts to fetch, even when those directory URLs have no inbound links and the server is configured to return a 403-Forbidden to directory requests when no index page exists in a directory -- a very common (and in many cases, default) server setting.
I spoke to Yahoo at the PubCon about this and he said they would look into it. The slurp isn't quite as agressive as it was but still continues to try and access fill in request.
Just this all started with the august update along with the improper reponse of long established 301's.