Welcome to WebmasterWorld Guest from 126.96.36.199
Forum Moderators: mack
If an MSN Live rep comes across this thread, do please look into some of these issues. From a webmaster's perspective, these little quirks are really getting annoying.
At a measly 6% of my overall traffic, I'm done wasting my time paying attention to MSN Live. Bill, what happened to your claims of "trouncing Google"? LOL
Addendum: Sometimes I wonder whether they take search as seriously as I'd think they would, given that Google is very close to dislodging them as the most important computer related company in the world. Have they conceded the importance of quality search to their competitor? Wouldn't think so, but I do wonder, given what we are experiencing, and their apparent failure to make timely corrections.
did MSN ever stated that they support wildcards and/or regular expression syntax in robots.txt?
And I do want to emphasize, until this latest weirdness they did more or less stay out. What is really strange is how almost all of the straight-forward html dropped out (which WAS well indexed), and the cgi pages came in. It was like a complete reversal! Very strange.
We have to face facts - Live search has been next to useless for years now, it has next to zero pull in the market, they treat webmasters with utter contempt, msn cant be bothered to visit this forum any longer nor are they interested in feed back from webmasters - its a total waste of our energy discussing live search here at WebmasterWorld.
My call is to delete this section and we forget about them in the same way that they have us.
You highlighted few issues and you have few questions:
About our communication:
Hope this helps.
To start with, it would be good to take note that since your latest update, it appears that a LOT of established sites have been knocked out of your index, for no apparent reason. That is not good.
Secondly, in my specific case, I again want to point out that my robots.txt file has had
User-agent: msnbotat a particular site for well over a year, but with this latest update you removed almost all the html pages and instead are displaying the very .cgi files that should be blocked. That too is not good and indicates definite problems. The things that are happening to us here at this board could very well be happening to hundreds of thousands of others as well (perhaps millions).
Traffic lost from MSN has been replaced by traffic coming from Ask - who seem to be trying to challenge Google's dominance with action rather than MSN's talk.
Many of the pages that now appear at the top defy all logic...many are nothing more than store fronts with little or no content. There can't be any meaningful testing going on before deployment.
I really wouldn't care except for one thing. Many corporations will only allow the MSN search on their computers. So when the Purchasing Department goes on line to do searches they're using MSN. Employees shop on company time and they're using MSN. It makes a huge difference when MSN screw things up.
But there is nothing to do about it right now. I agree to just stay out of it because "the rules" don't apply right now to MSN Live Search. It's a waste of time. We have supplied well-optimized, high-traffic websites with great content. Live Search just can't find them anymore. The ball is MSN's court.
#pages and files in this folder are not allowed to be seen by spiders, contains high resolution images
#page contains contents to UPS Rate look up Script with in my site, a pop window
#page has been disallowed for 4 years, specificaly to ban obusive, rouge bots, spider trap and is listed on first page in Live.com when site comand is issued
#Same as above
MSNDude this is something that your team should think of.
Pocketseo, if you read it carefully you would notice that nowhere MSN mentions word wildcards. The fact that they allow for exclusion of file types using syntax that looks like wildcards/regex cannot serve as proof that they support wildcards/regex.
This is not "syntax that looks like wildcards/regex" -- it is a wildcard:
An asterisk that substitutes for other characters is a wildcard. MSN should specifically mention on that docs page that they only partially support wildcards.
Yahoo and Google both support wildcards in robots.txt so MSN is not being clear in this case.