Forum Moderators: mack

Message Too Old, No Replies

MSNBOT indexing at the speed of light

A note about MSNBOT and how quick it indexes

         

stinkfoot

9:01 am on May 17, 2006 (gmt 0)

10+ Year Member



Well I cant believe it.

I put a site up and in 2 days it has been indexed and is MSN returning results for exact string searches on it. How quick is that? I mean .. google hasnt even touched it and Yahoo .. well .. nuff said .. blahoo more like.

With indexing speed like this why isnt everyone using it? New information gets in quick. Which is what is supposed to happen in a information rich enviroment like the internet.

Blow the others MSn rocks the others to the ground for speed on indexing.

milanmk

12:20 pm on May 17, 2006 (gmt 0)

10+ Year Member



Same here, put up a new sub directory based site and it was indexed by MSN in 5 days flat with good ranking for two word keyword. The same keywords show age-old results in Google and the top 3/10 sites are spam.

Moreover, if I add a couple of pages to the existing website than it hardly takes around a week time to index in MSN. I tried my best to get into Google SERPs but finally I gave up and started optimizing my website for MSN. Now I get around 75% of total traffic from MSN and rest from Yahoo.

Milan

steveb

7:32 pm on May 17, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Google indexes pages in 20 hours. MSN is still slower but they do alright.

Pages ranking well immediately is a bad thing, not a good one, unless the topic/query is brand new too.

F_Rose

7:37 pm on May 17, 2006 (gmt 0)

10+ Year Member



Our SERP'S on Microsoft are excellent. However, we have very little hits with Microsoft..

stinkfoot

9:39 pm on May 17, 2006 (gmt 0)

10+ Year Member



Steve,

>Pages ranking well immediately is a bad thing

I dont know that I agree with you there tbh. New information is important. Yes authority sites should also rank well but problems arise deturmining who is an authority site and who isnt.

If a goverment site has information on a given theme are they supposed to be the only ones that rank well because they are the authority? No imo.

When someone produces new information on a well versed subject should it be disgarded like it is in some search engines or given air like it is in better ones?

eg. you are the voice of propaganda and I am the voice of freedom. Who is the authority? The new voice should always get aired imo.

treeline

10:37 pm on May 17, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Sites ranking immediately makes it too easy for spammers. As soon as they get knocked off the top, they take a quick look at what happened, adjust/copy and are back on top. They can see in near real time if their ideas worked, and experiment and debug quickly and efficiently.

Anything exploitable gets found and overused quickly. Soon there's nothing but spam on the first few pages of SERPs. No, make 'em wait a bit and wonder. Really interrupts the efficiency of the abuse cycle.

steveb

2:51 am on May 18, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



There is hardly ever value in "new". Search engines that rank anything immediately of course end up ranking more trash immediately than anything else.

A good search engine will make you earn rankings, not rank you first because you put keywords in your subdomain.

Indexing quickly is good. But then do all the off site stuff to judge quality.

Marcia

3:12 am on May 18, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I believe this thread is about how fast MSN indexes, isn't that right?

Why just negatively ASSUME that a new site is trash? Some are highly valuable going out the gate first thing; no one can judge without seeing, nor can any kind of intelligent comment be made in ignorance. In fact there's plenty of trash out there with a spammed out link network, that's not any more a solid criteria.

That isn't what the topic is anyway, which started out positive enough, until the wet blanket was thrown on it.

Back on topic,

With indexing speed like this why isnt everyone using it? New information gets in quick. Which is what is supposed to happen in a information rich enviroment like the internet.

They're also very quick to pick up and index updates on existing sites, which keeps the index nice and fresh.

stinkfoot

1:52 pm on May 18, 2006 (gmt 0)

10+ Year Member



Marcia,

Yes this seems to be the case. New information that is updated on existing pages also does seem to get in quickly.

This can be good imo. Fast updates to existing pages in an index, showing new quality information, will leave the stale pages far behind.

With the power of MSN there is no reason I can see it not being used in today's world as a good research tool as well as for all of your holiday needs. (oops given the game away about my industry there haven't I)

For research new info is paramount. As for holiday needs up to date pages ranking highly means realistic prices and up to date reviews.

Go on Steve, throw that old mouldy stale loaf in the bin. There is fresh stuff coming at you if you just looked in the right places.

Just as a note I did actually say that my 2 day old page was ranking with an exact string search.

F_Rose

3:18 pm on May 18, 2006 (gmt 0)

10+ Year Member



I can't wait for Google to become history.

MSN is doing a great job, however they have very little searchers.

Can't wait for people to start using MSN, and forget about Google.

steveb

9:08 pm on May 18, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Indexing quickly is a good thing, but "indexing" has no comment at all on quality.

stinkfoot, I don't think you are appreciating that new can be just as "moldy" as old. Making sweeping assumptions like Marcia that everything is one way or another isn't going to help you. I could put up a new page today that has information from twenty years ago. Ranking that page because it is "new" on the Internet makes no sense. The newness of an html page has no direct connection to the newness, or the value, of the information on the page.

The value of fast indexing is that it gets a new page or site into the mix sooner, where it then must be judged in terms of its quality. New is of no value. Just because my sister says the same content today in different words than she has said before doesn't mean it is worth listening to. Value is of value.

treeline

9:36 pm on May 18, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Another downside to continual super-frequent re-indexing of pages is the bandwidth drain.

If a page includes news, weather, or some other constantly updating info it will always look somewhat different to a robot, even if only a little. A few spiders must be real lonely because they visit a lot. I don't need an essentially stable page indexed every hour just because the minor item of local temperature goes up or down a degree every hour.

Search engines that deliver lots of visitors (like Google) are easier to forgive for gobbling up bandwidth. MSN delivers a small fraction the visitors, but eats up bandwidth like MS Windows rakes in licensing fees. They need some sort of reality check.

crobb305

11:42 pm on May 18, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I also see msnbot spidering heavily these days; they are doing better. They just need to commit to getting rid of the subdomain spam. They can spider like mad and still have horrible search results. Sort of like rearranging the deck furniture on a sinking ship.

MLHmptn

5:44 am on May 20, 2006 (gmt 0)

10+ Year Member



Implemented mod_rewrite and 301'd every page on one of my sites. In one weeks time MSN has now updated 50% of this particular site. Yahoo, about 10%...Google....0%!

Of course this is just one of my sites but it speaks volumes in my eyes.

I also get highly discouraged when I hear that MSN's results are spammy. In the sector I'm in the top 28 results are all relevant! To me it's quite surprising that the SERP's I monitor can be so relevant without human intervention/placement. Google comes nowhere close to that, nor does Yahoo.

My hat is off to you MSN! I have marketed MSN search on my websites for some time now and I'm happy to promote MSN search to every one of my visitors! Honestly I would like to put a note next to the search form that directly tells my users to check out what Google is hiding from them! Of course that wouldn't be too professional of me to do so I haven't done so but don't think I haven't thought about it.

BillyS

9:12 pm on May 20, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Six days ago I put on my site a new tool for users. MSN is the only search engine that's picked up the new page and is showing it in SERPS.

MSN is fast.

steveb

1:53 am on May 21, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I put up a page 25 hours ago, and linked to it with VERY unique link text. (The word wasn't on my site before this.)

Google had the new page in the index after nineteen hours, and now has seventeen pages(*) in their index with the unique text. MSN has zero. Yahoo has zero.

I expect MSN to pick up the page in a couple days, and at least one of the linking pages by tomorrow.

MSN likes some sites better than others. Google likes some sites better than others. (Yahoo is more like "Mikey" and doesn't like anything much...) There isn't much sense to comparing how fast they index one single site. MSN seems to have a day and a half turnaround these days, while Google is under a day... when they each "like" a site. No big deal in the difference there, but don't go thinking what happens with one site is what goes on with others.

(*)About 23 pages have the text in total.

steveb

9:48 pm on May 21, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Twenty-one hours later, MSN has the new page indexed, plus two other pages show the text. Yahoo still zero. Google still seventeen pages.

nakulgoyal

11:26 pm on May 21, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



a brand new website ranking in top 10 for a 100k searches a month keyword.

stinkfoot

1:24 pm on May 26, 2006 (gmt 0)

10+ Year Member



>Twenty-one hours later, MSN has the new page indexed, plus two other pages show the text. Yahoo still zero. Google still seventeen pages

Nearly a week later and 2 hits from google all pages hit by msn bot 5 or so by Yahoo.

Steve your pages are obviously on a well indexed site that is quite old with a sitemap attached ... new sites dont have those advantages .. but they do have msnbot!

trinorthlighting

3:49 pm on May 26, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Yes, msn has the search engine business down. It does index faster than google.

Also, msn is very relevant for specific keywords. Google is still stumbling around with index issues as always.

I do not know how google ever expects to outpace and win over bill gates and his company. Bill beats them all in the long run......

steveb

7:18 pm on May 26, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



No, the point is some sites are liked better by one bot or another. Saying one engine is faster than another is subjective. I do know that I've never seen anyway post MSN gets pages in in less than 24 hours, and so is certainly not faster than Google, but that isn't some critical thing.

Google has had all 24 pages for several days now.
MSN now has six (down one).
Yahoo now has eight, but NOT the new page itself, only pages linking to it.