Welcome to WebmasterWorld Guest from 54.167.5.15

Forum Moderators: mack

Message Too Old, No Replies

Microsoft MSN Bot Live in the Wild!

Microsoft is Crawling

     

bolitto

2:02 am on Jun 17, 2003 (gmt 0)

10+ Year Member



Look who's all over my sites!

"MSNBOT/0.1 (http://search.msn.com/msnbot.htm [search.msn.com])"

I've checked the IP - it's legitimate, Microsoft is crawliing the web!

IP 131.107.137.xxx

msr986

2:15 am on Jun 17, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Others have seen the MS bot before, but I have not seen the information URL.

Previous threads
[webmasterworld.com...]
[webmasterworld.com...]

martinibuster

2:16 am on Jun 17, 2003 (gmt 0)

WebmasterWorld Administrator martinibuster is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



This is cool, good find. At last they have their bot faq up.

Stefan

3:35 am on Jun 17, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member


Read that 7 page thread, then went to http://search.msn.com/msnbot.htm

So it's decided that it's legit? It looks that way but I'm starting to get fuzzy...

Chicago

4:30 am on Jun 17, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



MSN results are actually looking better today than on G.

dvduval

4:48 am on Jun 17, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



MSN results are actually looking better today than on G.

Are you part of the "trustworthy computing" campaign? :-)

Once I got past the pop-up window I found the reslts to be...well...I better stop before I say something I regret!

martinibuster

6:01 am on Jun 17, 2003 (gmt 0)

WebmasterWorld Administrator martinibuster is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Once I got past the pop-up window...

dvduval, what url are you typing, and from what part of the world? I do not receive a pop-up.

Let me guess (and you be honest!) are you a Mac loyalist?

:) Y Y

Fiver_321

8:09 pm on Jun 17, 2003 (gmt 0)



Are they crawling with MSN bot because they wont be able to use inktomi soon - now that Yahoo owns ink?

heini

8:54 pm on Jun 17, 2003 (gmt 0)

WebmasterWorld Senior Member heini is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Hey hey - so we are looking at 4 major players in the websearch game soon?
G -- ATW/AV/OV -- Y!/INK -- MS

dvduval

8:59 pm on Jun 17, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hey hey - so we are looking at 4 major players in the websearch game soon?
G -- ATW/AV/OV -- Y!/INK -- MS

I wonder how long it will take people to actually start searching the others again now that they are so conditioned to use Google.

I would guess Yahoo has the greatest chance of success, followed by MSN. I love ATW, but I question how easily it can become widespread.

jeremy goodrich

9:01 pm on Jun 17, 2003 (gmt 0)

WebmasterWorld Senior Member jeremy_goodrich is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Microsoft has published some incredible research in collaboration with a university in Beijing, China.

The interesting bit is that they were looking at other types of correlations than the ones that the other players look at to map 'relevance'.

Another interesting bit is that Microsoft is one of the first large US corporations to incoporate fuzzy systems into their products, and that China has the world's largest number of fuzzy scientists.

Google's technology is based on a probabilistic model which is a subset of fuzzy logic, as determined by 'probability & the hypercube' by a professor named Bart Kosko.

In my mind, better math = better engineering. If I were Google, I would seriously rethink my bias against non traditional systems & fuzzy logic, and toss the binary thinking that may constrain their ability to return optimal web results.

dvduval

9:05 pm on Jun 17, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Jeremy,

When you talk of departing from binary thinking, do you mean displaying dynamic results that can vary on a page refresh?

jeremy goodrich

9:55 pm on Jun 17, 2003 (gmt 0)

WebmasterWorld Senior Member jeremy_goodrich is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Try reading through a bit of MS history / engineering / and their papers out of the Beijing University I mentioned (might have posted a link here, not sure).

Google in their research, continually reference probability, as the solution to deal with ambiguous or incomplete data. They have an incredible search engine, but at Stanford, even the current research on 'bettering PageRank' involves using a different approach.

I love their engine, however, when you read through their old papers, it's pretty clear what math school of thought they were following.

The stuff that MS has been doing for years, is only possible with fuzzy systems...applied to the web, and you get math in 3d instead of 2d.

Brett_Tabke

10:01 pm on Jun 17, 2003 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



> MSN results are actually looking
> better today than on G.

Some days, there ought to be a license to compute ;) Put the keyboard down and step away from the computer. lol!

>131.107.137.xxx

So is it just that ip block? That is one for the htaccess list for sure.

Hardwood Guy

10:09 pm on Jun 17, 2003 (gmt 0)

10+ Year Member



I noticed some increases in MSN traffic the last few days and wondered myself what was going on.

1 www.google.com 43.0%

2 search.yahoo.com 19.7%

3 search.msn.com 18.9%

4 aolsearch.aol.com 6.1%

Two weeks ago...

1 www.google.com 43.5%

2 search.yahoo.com 32.2%

3 search.msn.com 11.3%

4 aolsearch.aol.com 4.3%

Stefan

10:15 pm on Jun 17, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Well, so sign of the beast in the logs for yesterday... I'm in two ODP cats, the thing should show up soon I hope. As much as I detest Bill, I'd hate to be left out of the party.

Is anyone seeing it being particularly active yet?

Brett_Tabke

10:30 pm on Jun 17, 2003 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



>Is anyone seeing it being particularly active yet?

Yes, they are seeking out SEO's to beta test (they call you - don't call them). However, the NDA has more pages than webmasterworld I believe.

The only question left is if it will run Windows or more probably on Unix or an IBM mainframe.

Stefan

10:44 pm on Jun 17, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Man, I guess I'll have to read through a lot of archived threads...

So how will this work? Once they have a large enough index from the new bot, they dump Ink and go with their own data?

(NDA - National Dance Association. A fine bunch, by all reports)

awcabot

11:07 pm on Jun 17, 2003 (gmt 0)

10+ Year Member



I do not know if this has been noticed before, but MSN is looking for search engine developers. See Job Opportunities with MSN Search [search.msn.com].

I guess they could not buy Google so now...shudder the thought...

martinibuster

11:21 pm on Jun 17, 2003 (gmt 0)

WebmasterWorld Administrator martinibuster is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



The newly expanded MSN Search team is working on indexing the entire internet and returning best-of-class results to search queries.

Wow.

Think they'll do a PFI?

I wonder if internet search will be integrated with Longhorn.

IE is suspended [ecommercetimes.com] as a standalone product, and the next iteration will be as an integrated aspect of the Longhorn OS.

[edited by: martinibuster at 11:28 pm (utc) on June 17, 2003]

john316

11:26 pm on Jun 17, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



My guess is that the index will be an integral part of the new "we don't have a browser bundled, because we don't call it a browser" OS.

Search integrated with Word...

Why change tactics now?

Mozart

11:57 pm on Jun 17, 2003 (gmt 0)

10+ Year Member



Great, a new robots.txt will go up right now :-)

User-Agent: MSNBOT
Disallow: /

...and then I wait for MS to not obey it. I just can't imagine them getting THIS right!

If you folks out there would like to help me: Simply use above robots.txt and post the first time the bot does not obey. I'd love to see that!

Sorry, but Bill has the rights to pictures of Marilyn Monroe and the Moon Landing, he really does not need to tell me what to find when I am searching the web!

And about losing a few bucks in say the next year because Bill has integrated the search into the OS: "They can take our bucks, but they can't take our FREEEEEEEDOM!"

panic

2:59 am on Jun 18, 2003 (gmt 0)

10+ Year Member



If I'm not mistaken, the search is already integrated into the OS. There's two ways that I know of to search the web using MSN straight from Windows :

1) Simply type your search term into the Internet Explorer address bar.

2) Start Menu -> Search -> On the Internet

-panic

martinibuster

3:27 am on Jun 18, 2003 (gmt 0)

WebmasterWorld Administrator martinibuster is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Amadeus,

I have seen a number of your posts around, and I respect your opinion- But I have to question your last post.

Yes, they have the rights to a lot of photos. Yes they make a lot of money. Yes MS has a ubiqitous presence in computing and finance. Yes they're a bunch of dorks when it comes to marketing. Yes, because of their size and the abundance of MS Hacking Tools they're the target of every angry overprivileged teenage hacker...

But is this really a reason to be resentful of MS if they want to get into the web search business? I say no. It's not logical.

Think about this: Visit the Bill and Melinda Gates Foundation [gatesfoundation.org] web site and see how they've donated over SIX BILLION DOLLARS in 2003 to worthy causes like fighting aids in Africa, education, libraries, etc.

What has Apple done besides self serving donations of computer equipment to design schools in the 80's?

Has Google behaved as a responsible citizen of the world and given back even a fraction of a fraction of 1 percent to make life a better place for those less fortunate?

Please don't be confused: I am not positing Microsoft's charitable practices versus Apple or Google.

Perhaps this is a little off-topic but, with great respect for your opinion, so is your post. And that's the point of this post: It's important to keep to the topic.

I'm not discounting the value of healthy skepticism but your post goes beyong skepticism and, no disrespect to you intended, it delves into the realm of knee-jerk reactionism- which does nothing to further the discussion of a Microsoft Bot in the wild.

But I agree with you Mozart, they can take our bucks, but they can't take our freedom.

I applaud MS's foray into search because it has the potential to give Google some competition, and that can only be good for everyone because it will drive innovation.

[edited by: martinibuster at 3:40 am (utc) on June 18, 2003]

Macguru

4:56 am on Jun 18, 2003 (gmt 0)

WebmasterWorld Senior Member macguru is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Back to topic, they seem to have little adjustments to do about clustering. I have results from the sames domains repeated several times in first 3 SERP. Is it an clue they are mixing results from several sources?

keyplyr

7:17 am on Jun 18, 2003 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Saw MSNBOT today for the first time. Crawled about half our pages, requesting robots first off.

littleman

8:53 am on Jun 18, 2003 (gmt 0)

WebmasterWorld Senior Member littleman is a WebmasterWorld Top Contributor of All Time 10+ Year Member



How could MSN not have a search engine powered by MS technology? It must be surely embarrassing to them that their search results are powered by open source OSs.

richmc

10:19 am on Jun 18, 2003 (gmt 0)

10+ Year Member



Saw MSNBOT today for the first time. Crawled about half our pages, requesting robots first off.

out of interest keyplr, how many pages roughly was that?

Brett_Tabke

10:37 am on Jun 18, 2003 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



> search results are powered by open source OSs.

The first net engines showed up in 94, and there are historical firsts in db/se programs long before that.
And it has taken MS until 2003 to do something? The worlds largest software manufacture with 20k'ish programmers. hmmm speaks loudly of the ability of Windows to scale (lack thereof).

ncsuk

10:45 am on Jun 18, 2003 (gmt 0)

10+ Year Member



Can you say that?
This 75 message thread spans 3 pages: 75
 

Featured Threads

Hot Threads This Week

Hot Threads This Month