Google Says MUM AI Model Is Changing Search

[engine]

There's a whole range of announcements out of Google and here's a quick summary.
Google's MUM (Multitask Unified Model) has reached a new milestone and Google is describing the new ways to search coming in the next few months. Google Lens will allow users finding an item to use Lens to find similar products that might be difficult to describe.
It'll also include "Things to know" to help users find other similar topics of interest. In a drive for e-commerce, it'll also show more stock inventory in Shopping results. For example...



It'll also include "Things to know" to help users find other similar topics of interest, and this is launching in the next couple of months. For example.


In the coming months it'll also be releasing the ability to "zoom in and out of a topic with new features to refine and broaden searches." For example..


It says it's also going to make finding inspiration easier with a "newly designed, browsable results page."

Videos in search have been around for a while, but with MUM it'll be identifying related topics in a video, and topics not explicitly mentioned in the video.

As we continue to build more useful products and push the boundaries of what it means to search, we look forward to helping people find the answers they’re looking for, and inspiring more questions along the way.

[blog.google...]

[Sgt_Kickaxe]

I think you can see "MuM" at work in any video by turning on closed captioning. Where the CC used to try to closely match what was said now it ad libs a bit.

example: When the maker of the video says something the CC displays "Mike says: yadda yadda", The video never mentions Mike but MuM has identified the voice.

I noticed this on a video made and narrated by Dave so "Mike" stood out, the CC guessed wrong.

Besides names I noticed the CC attributing words to "the worker:" and " an employee:" but the videos did not say that one person was a worker or the other an employee... these were guesses.

MuM's been busy. Where exactly is it sourcing it's guesses on video CC? Are all content creators being profiled?

[engine]

It's a little easier for this to work on static imagery as software nowadays can do quite a good job of identifying the image contents. It's no way perfect, but it's not too bad.

When it comes to video, again, it's working hard to identify content and describe it as best it can. It's finding "key moments" reasonably well. Still early days, of course.

On speech syntheses, the technology is relatively long on the tooth, but has reached a plateau, imho. It's literal, and understanding meanings is quite complex: Even humans fail at that through misunderstandings.