Loss of Default Site Page in MSN Serps

[SEOmariachi]

I have had an odd problem develop over the last 72 hours in relation to MSN Search. I have a few sites that, even though their default pages are getting crawled multiple times a day by MSNBot, do not have their default pages indexed by MSN Search anymore. Each site has at least 500 pages still in the indexed, and is still getting crawled just as much as they were a week ago. I think that the default pages for the site ranked very well in MSN because of the inbound link counts for these sites, but now they are no where to be found on the Serps (which has greatly limited my traffic from MSN).

Any Ideas?

(I am still getting traffic from MSN on 3 and 4 word terms, so referrals have not died entirely).

Thanks in Advance,
SEOMariachi

[ssjothun]

We have the same problems at Bodog, and we've had it for a while - pages just completely disappear from the index, while the rest of the site with over 10 000 pages still there. This time it was our NFL Lines page, but it has previously been the Poker main page for example.

Does anyone have any idea what's going on?

[ssjothun]

Ok...this is really strange. From reading this thread [webmasterworld.com...] it seems like most gets their index page removed from the index while the pattern for us is that the deeper main category pages gets removed.

It also seems like it's only the pages driving massive ammounts of traffic to the sites that are being removed.

msndude mentioned in the referenced thread that they are testing some new spamfilters. The problem is, there is no spam, and the entire site is all white hat(probably the only site in this industry...lol)

Any feedback or ideas would be highly appreciated

[steveb]

MSN regularly drops pages from its index. That is what they do. That is part of their deliberate plan. They choose to not fully index sites. Them sometimes dropping critical pages instead of merely deep content internal pages shouldn't be too much of a surprise.

[ssjothun]

No offense, but I think it's a bit far fetched to say it's a deliberate plan.

What is a fact is that they have a neural network that's still learning so to speak. Another fact is that they are constantly adding pages to their index. That is what they do, and that is what's in their best interest.

Dropping pages deliberately from the index is just bad business. Even MS wouldn't do that.

[stu_uk]

I agree, i dont see it as being a deliberate move, im far more inclided to think they are testing out different filters.

As a side note it quite possible for white hat site to suffer OOPs, especially if its a site that relies on input from users e.g. a forum. If one of your keywords is naturally at a fairly high density and then a few users come along and mention it one too many times it can easily trip a filter.

[ramachandra]

Yeah, many of site index pages vanishes from msn index again will be back in index within one or two week, this happen for couple of my sites recently which is now back.

[ssjothun]

Our "nfl lines" page is back today and we're back on page 1 again, however just #10. But even a day's absense in the SEPRs is not good, and not acceptable IMO.

We did contact MSN about this problem yesterday, and I'll keep you guys updated.

[ssjothun]

And oh my God, our Online Poker main page is gone today!

Not only that - I did a recount of our indexed pages, and it has dropped from 21 338 yesterday to 14 383 today. That’s almost 7 000 pages gone from one day to another!

[steveb]

"Dropping pages deliberately from the index is just bad business. Even MS wouldn't do that"

Sorry, but that's like saying the sky isn't blue. They obviously are deliberately dropping pages. It is obvious both because they crawl pages but still remove them from their index, and because of the phenomenon discussed many times here about pages a certain amount of clicks from the main page not ranking at all.

It's absurd to think that there is some sinkhole or something that accidentally requires MSN to accidentally drop 400 pages of a 1000 page site even though they crawl all 1000 pages more than once a week.

They have a clear policy to deliberately churn their database, quite apparently aimed at dropping deep content pages, but it should not be surprising that they sometimes drop key pages that they would not normally intend to.

[ssjothun]

And what exactly, from a business perspective, would be the purpose of that?

I mean, isn't it in MSN's best interest to make sure that their "new" search engine, that they have spend billions of dollars developing, delivers the best possible results? To make people prefer their search engine over others by delivering better results?

They don't even have their Ad Systems for MSN Search up yet(Microsoft adCenter Incubation Lab won't have it up before June), so you can't say they are doing this to make companies spend dollars on their paid listings either?

Again, no offense, but think about it - there is, as far as I'm concerned, no logic behind your statement.

[steveb]

"isn't it in MSN's best interest to make sure that their "new" search engine, that they have spend billions of dollars developing, delivers the best possible results?"

Again, it is plainly obvious this is not the case, as they geolocate results for terms that have absolutely no relationship to geolocation, like: neptune

Besides that, how exactly can you explain how virtually no domain over a few pages gets 100% indexed, despite all pages getting crawled, for a year?

C'mon now, you can't just ignore the obvious facts. They are right there for everybody to see. The motivation may not always be clear, but the practices are on display every day, and have been discussed here for months and months.

[RichTC]

Steveb

- You hit the nail right on the head!.

Agree 100%, Currently the search bot can not index large sites, its as simple as that. You only have to use the site command on msn for any large site and you can quickly see that the site has very few pages indexed.

Knock together a two bit site of say 50 pages in a week and bingo a week later the whole site will have been indexed.

msn search = still in beta stage

[ssjothun]

Ok...there's a slight difference between being incapable of crawling pages properly cause you have a new search engine, and a neural network that needs to learn, and not crawling them on deliberately.

Give me a reason, a motivation why MSN would do so deliberately, cause right now, I don't see any...

msn search = still in beta stage...now that's something I can fully agree on.

[followgreg]

MSNdude stated that they were testing for the next couple of weeks and that we should expect some weird things.

however one needs to explain me how dropping homepages helps?
Because this is what happens...and that's not good, is it?

If they have the same problems Google has - they won't go very far in the competition unfortunately

[dodger]

My homepage is dropped regularly as well, always comes back after a while. Pages indexed is pitiful.

MSN is a fool to itself and a burden to others.

[followgreg]

>>Dodger: I think that I will start thinking what you say if they can not fix their situation soon.
I still have a strong hope that MSN catches up with GG because it will be very exciting (IMO for Y! it's game over).

But BS like these, dropping homepages... are not helping...