Google : Rethinking Search: Making Experts out of Dilettantes

[engine]

Google has published a paper in which it argues that search as it stands today may not be the search of the future. The current model requires ranking to play a part in providing an answer, and then places "a rather significant cognitive burden on the user." Today's search services, including Google are wanting to provide answers instead of purely ranked results.Google goes on to say that this has had limited success.
In addition, is explains that there has been much progress into natural language understanding, including "word-embeddings", "sequence modelling, large pre-trained language models which capture relationships between entities.

This paper envisions a unified model-based approach to building IR systems that eliminates the need for indexes as we know them today by encoding all of the knowledge for a given corpus in a model that can be used for a wide range of tasks. As the remainder of this paper shows, once everything is viewed through a model-centric lens instead of an index-centric one, many new and interesting opportunities emerge to significantly advance IR systems. If successful, IR models that synthesize elements of classical IR systems and modern large-scale NLP models have the potential to yield a transformational shift in thinking and a significant leap in capabilities across a wide range of IR tasks, such as document retrieval, question answering, summarization, classification,recommendation, etc.

When you read this paper, it's quite clearly proposing the idea of an alternative way of searching and answering. Instead of document retrieval from an index, it's discussing pre-trained language models, model-based information retrieval, and even beyond language models.

If all of these research ambitions were to come to fruition, the resulting system would be a very early version of the system that we envisioned in the introduction. That is, the resulting system would be able to provide expert answers to a wide range of information needs in a way that neither modern IR systems, question answering systems, or pre-trained LMs can do today.Some of the key benefits of the model-based IR paradigm de-scribed herein include:
•It abstracts away the long-lived, and possibly unnecessary,distinction between “retrieval” and “scoring”.
•It results in a unified model that encodes all of the knowledge contained in a corpus, eliminating the need for traditional indexes.
•It allows for dozens of new tasks to easily be handled by the model, either via multi-task learning or via few-shot learning, with minimal amounts of labelled training data.
•It allows seamless integration of multiple modalities and languages within a unified model.

Here's the paper (PDF) [arxiv.org...]

[JS_Harris]

Interesting

[edited by: JS_Harris at 5:02 pm (utc) on May 23, 2021]

[JS_Harris]

P.S. Using the K.I.S.S. method for solving complex questions would be simpler.

What method is that? The one where Google tells webmasters what people actually search for. Tell a webmaster what questions are being asked and I guarantee you those pages will be created. We have brains too ya know :)

"What'as the difference between lakes, rivers and oceans?" was their example, if that ever turns up as a P.A.A. question it will be answered, ad-nauseum.

Google's secrecy handcuffs webmasters into not providing the content people want, the best we can do is pay for services that monitor google and make best guesses. Imagine if Google shared more specific data about what people looked for....

[brotherhood of LAN]

Agree with others that it's just another land grab by Google on the thought space of search .Nothing to worry about, pah.

CTR to the organic web has steadily decreased over the years. DMOZ once heavily relied on, surplus to requirements. This kind of research is the same outcome for wikipedia, one less thing for Google to rely on. They can obfuscate their reliable sources into a model and deny where the information reliably came from.

The reduction in CTR is maybe admissable for universal truths like "what is the time in London", there's no reason to click through to anything else, there's no reasons for websites to be dedicated to answers that can be universally answered with a search where there's only one answer - I can go with that.

I saw an interesting comment about knowledge graphs and how they're doomed to fail at scale and that 'perspectivism' is the way forward. I'd tend to agree. Google can do what they want, as long as there's other perspectives into the organic web other than their own, that aren't shoved out of the market by the monopoly that G basically has.

I'd rather the 10 blue link perspectives on subjective questions than something 'ontologised' as truth. It's more a question of philosophy of knowledge than anything technical.

[NickMNS]

Let me just veer off topic slightly or digress. The one message that is clear from this is that as webmasters/business people I think future strategy is clear. There is no longer a business case for creating linear or static informational websites, where one simply provides content eg: like a book. Instead information will need to be provided to the user in a more interactive or dynamic way, for example as visualizations or customized. This will make it impossible for Google to use the information to feed it's systems and will make the website content more appealing for the user.

It of course creates new challenges, how can Google send you traffic without you revealing your content to Google? Catch22!

[JS_Harris]

OK, now that I've researched all of this a bit more, I'm leaning towards the approval side of where things are going.

The MUM paper especially, information not being constrained by language, queries becoming more complex and nuanced but still providing good results... I like it. I'm not sure how Google plans to implement it so it very well might be the end of traffic from them but the benefits are there, if they don't kill off websites in the process.

At least big corporations won't be able to toss $5 at a content creator to create a paragraph of text for this query and that anymore. Not with great effect anyway.

I am curious however if the new systems will simply become all of the infomational content and only require shopping intent sites when the user clearly wants to buy. I mean, if the person asking speaks a different language than the site the information came from Google can't exactly send the user to a foreign language site. Local store, sure, but not the informational site answering the question.

Do informational sites need to become multi-lingual asap?

[iamlost]

Do informational sites need to become multi-lingual asap?

Not necessarily.
What it means is that rather than currently where G pulls results for an, for example, English language query solely from an English language site they can pull from any number of any language(s) site(s) while rendered by the results in English.

As providing such an English query a link to the best answer site if not an English site is largely counterproductive for the user this methodology is obviously best when no such link out is provided, just an answer on G.

Of course, G could link to a G cached English site translation but that seems cumbersome or to on fly English translation but seems contrary to current trajectory.

[ember]

The current model requires ranking to play a part in providing an answer, and then places "a rather significant cognitive burden on the user."

Oh, no. Requiring people to think? We can't have that.

[FranticFish]

Little did I know the 'rather significant cognitive burden' I bore during the 00s and 10s when I was required to evaluate multiple answers to my questions was so onerous. Thank goodness someone is prepared to release me from the agony of picking my preferred answer to a question. Who needs to worry about subjectivity when machines understand it so well?

[iamlost]

Options? You don't need no stinkin' options!


Searcher: Let me see the search result options.
Google: You don’t need to see options.
Searcher: I don’t need to see options.
Google: This what you need to know.
Searcher: This is what I need to know.
Google: You can go about your Google Shopping.
Searcher: I can go about my Google Shopping.
Google: Follow Google directions. Move along.
Searcher: Following Google directions, moving along.

[martinibuster]

>>>CTR to the organic web has steadily decreased over the years.

No. Those articles were 100% misleading click bait. The claims being made about zero click searches are 100% wrong. The opposite is true.

I am very tired of the poorly researched information being spread by certain people in our industry.

I'm just trying to help, for your benefit, by pointing out the facts.

"We send billions of visits to websites every day, and the traffic we’ve sent to the open web has increased every year since Google Search was first created.

…we’ve seen that as we’ve introduced more of these features over the last two decades, the traffic we’re driving to the web has also grown — showing that this is helpful for both consumers and businesses."

[searchenginejournal.com...]

[edited by: martinibuster at 3:39 am (utc) on May 27, 2021]

www.searchenginejournal.com Google: Zero Click Claims Are Misleading Google offers convincing evidence that claims surrounding "zero click" searches are erroneous

[martinibuster]

Every year when Google rolls out a new feature, the clickbait people come out yelling that the sky is falling and it never does. Never.

Google published the Dilettantes paper saying that they have to figure out how to answer Long Form Question Answering. BUT they haven't said how they will implement it or in what context.

Google hasn't released anything, they're still figuring out how it will fit.

[NickMNS]

@Martinibuster
From the article you linked as support of you argument (your own article no less)

The author (Google Search Liaison Danny Sullivan) also raised the issue that people use search differently than in the past and that can result in search queries that require an instant answer but do not need a click.

and later:

Google’s Danny Sullivan offered as examples...
People look for quick facts

From the horses mouth. The point being made here is that these searches are ending at Google, but the data to populate these "quick facts" are originating from content creators. The technology in question will make it possible to further increase the volume of queries for which "quick facts" can be provided and as a result content creators will see less traffic. No one cares about Rand Fishkin and Bobo-Zero or any of the other of the pseudo-science studies and tools that this industry produces. It is simple economics, Google is extracting rent. The more queries that it can provide "quick facts" for the more user don't leave Google.

You can try to pull the wool over your eyes and convince others but in the end anyone in this business see the impact the Google and Facebook are having. Just look at the state of the news rooms around the world. That industry is not on the brink of bankruptcy because Google is sending the them more traffic. Stop drinking the koolaid.

[Wilburforce]

We send billions of visits to websites every day, and the traffic we’ve sent to the open web has increased every year since Google Search was first created.

Estimates vary - as does the number itself - but somewhere in the order of half-a-million websites are created daily. Existing sites produce new content, and with it new pages.

We don't know the rate of increase of Google clicks, but if the rate of increase in page-numbers is greater, then all else being equal each page will get fewer clicks.

All else is not equal, and good content may see an increase in clicks, but if it is new content, those clicks will come from the total population of clicks, meaning another page will get fewer clicks.

The number of searchers is also increasing - increasing the number of potential clicks - but although this increase is substantial, it is slower than the increase in pages.

The overall trend must therefore be that - whatever Google does - there are proportionately fewer clicks to go round, as the number of destinations for clicks is increasing faster than the potential number of clicks.

It is also quite likely - proof in any individual case is another matter entirely - that independently of those source trends, Google changes may substantially reduce the number of clicks to a previously popular page.

It is more likely that quoting statistics - actually, not even statistics, just claims based on unknown data - without looking at underlying source trends will misrepresent reality.

[martinibuster]

>>>The point being made here is that these searches are ending at Google...

The point is that people search differently.

People are searching for businesses near them, phone numbers of businesses, and directions to businesses. Those kinds of searches didn't exist in the past.

People are also asking about film and TV show cast members. That's knowledge graph information. Those aren't stealing information from anyone's websites.

People today search differently and the searches continue to evolve. THAT is the point.

The wool that's been pulled over people's eyes comes from those who would have you believe the lie that Google is stealing clicks. The lies come from certain click baiting entities in our industry who have a long history of promoting fake stories.

>>>>The overall trend must therefore be that...
Sounds reasonable but you're dismissing what doesn't confirm to your pre-existing belief and rationalizing ideas (not facts) to shore up your belief.

The facts are that people search differently today. We do not live in the Ten Blue Links world.

I went though a similar experience with this forum in 2012 and 2013 trying to convince you all about user intent and other changes in the SERPs and you people dug in your heels against reality and insisted nothing has changed, just keep on hammering those keywords like it's 2003.

It feels like it takes about ten years for people in this community to grudgingly accept change and by the time they do the world has changed again and you're still ten years behind.

I'm trying to help you people get past those lies and misleading information. But I don't know, seems like your nostalgia is unshakable.

[brotherhood of LAN]

>The wool that's been pulled over people's eyes comes from those who would have you believe the lie that Google is stealing clicks. The lies come from certain click baiting entities in our industry who have a long history of promoting fake stories.
>People are also asking about film and TV show cast members. That's knowledge graph information. Those aren't stealing information from anyone's websites.

[docs.house.gov...]

Not sure about the poor research and lies you're talking about, I don't follow the SEO crowd.

A bit OT anyway as the above example is particular datapoints rather than longer tail.

[martinibuster]

>>>>[docs.house.gov...]

The guy in that document was scraping information about celebrity net worth and republishing it. Very shaky business model, the thinnest of content. THIN CONTENT.

And he's complaining that Google is scraping it from Wikipedia?

I just searched for the query mentioned (Larry David Net Worth) and Google has a featured snippet with a link to a website, an ORGANIC search result that leads to a website.

Did you read the document? The guy was publishing an entire website based on thin content. What is the take away from that link?

[brotherhood of LAN]

>Larry David Net Worth

What do you see for Larry Page Net Worth?

>take away

You must have missed the part where Google asked him for an API, he refused, so they scraped his site anyway - including fake listings designed to catch them out.

There's also the part where he talks about traffic decreases.

[martinibuster]

Hey BoL, I appreciate your engagement in this discussion, just want to let you know I appreciate your input.

>>>so they scraped his site anyway

Scraping is not the correct word. The use of that word in the context of normal indexing of a website is misleading. It's called indexing.

Google indexed his site and displayed his content in the Featured Snippets WITH attribution, just like they do for everyone else.

Some of his content was apparently syndicated to other sites (or scraped) and some of that content was shown in the featured snippets, too.

If that content was scraped, that's on him for not blocking the scraping and it's not like he didn't have recourse against a site like BankRate.com

Many of the links went to other sites, like the mortgage referral site Bankrate.com, even though those sites cited CelebrityNetWorth as their source.

There is no take away there.

The guy got indexed in the normal manner. If he didn't want Google indexing his site and displaying his content at the top of the SERPs in the featured snippets he could have blocked them.



[theoutline.com...]

theoutline.com How Google ate CelebrityNetWorth.com Google’s Featured Snippets are not only often wrong, they’re also damaging to small businesses that depend on search traffic.

[engine]

It feels like it takes about ten years for people in this community to grudgingly accept change and by the time they do the world has changed again and you're still ten years behind.

I tend to agree with MB. With this document we need to look forward, not back.

Pretty much everything about the Net has changed in the 25+ years i've been involved, and there's no reason why it won't continue to evolve.

Before the Web and search, there were different forms of marketing and promotion to get the word out, and the click-equivalent was, for example, a reply to a direct mail piece, or an in-bound phone call from an advert in a magazine or newspaper. That's all changed because of disruptive technology, the Internet.
When the Web first started the earliest of search services were indexing and displaying what it'd found in its index from submissions or crawling. We had the Ten Blue Links. Each business producing an index had to monetize it somehow. It's history as to how Google, and others have done that.

For years now there have been alternative ways to search, such as voice. It mostly relies on the data in the search index.
Then there's social media, video, apps, etc.

This document is about the way after the index, and could end up being an entirely different search service running in parallel. As I said earlier, Google will want to monetize it, whatever it ends up being. There's a long way to go.

Getting back to my opening remarks in this post about change; as marketers we should be looking forward to the new and alternative ways of getting our message out there. If it doesn't generate "clicks," so be it.

We all have to be creative and to look at any emerging technologies and ways that may help, and even returning to some of the old fashioned ways of writing a letter.

[NickMNS]

@MB

Scraping is not the correct word. The use of that word in the context of normal indexing of a website is misleading. It's called indexing.

Absolutely if Googlebot does it is "indexing" and its ok, but if another bot "indexes" your content and re-uses it on their site its spam. Got it.

There exists a tacit agreement between webmaster's and Google, we (webmaster's) allow Google to scrape our content and to add it to their index. In exchange we expect that when users are using Google to search for topics relevant to our content they will be provided a link and a brief description of the content, allowing the user to visit the website.

Google has progressively and overtime eroded this agreement, changing a "brief description" to a "snippet" to displaying the full answer to the user's query such that visiting any website becomes redundant. Yes, to your point, to some extent for some queries it is reasonable to suggest that visiting a website to get an answer to a simple question is annoying and pointless. But again, this has eroded over time, and the basic queries, like convert 1 mile to 1 km has morphed to ever more elaborate queries and answer's. The technology described in this paper continues to push the limits and further erodes the agreement, essentially it pushes Google "indexing" ever closer to "scraping".

To be clear, I'm not suggesting that "Search" has not changed over the years and that we should go backwards. I'm the first to argue that the concept of "keyword" is dead, and to suggest that relying on 3rd party tool is nonsensical waste of time and money. But to suggest that we should blindly trust Google, and that any technological innovation that is good for Google is good for us is just as nonsensical.

[iamlost]

I have no regret regarding my initial concerns regarding probable (IMO) future implementation of the model described in this paper. And I see no reason to withdraw or even mitigate those concerns.

Emily M. Bender (Professor, Linguistics, University of Washington; Faculty Director, Professional MS Program in Computational Linguistics)

Modern search engines are an excellent example of human-in-the-loop: The human crafts a query, gets a ranked list of candidate documents to peruse, either finding what they're looking for or issuing a new query.

By clicking through to the underlying documents, the human is in a position to evaluate the trustworthiness of the information there. Is this a source that I trust? Can I trace back where it comes from? Is it from a context that is congruent with my query?

She is not alone, in academia, in taking issue with the underlying ‘problem’ that the paper purports to address.

Further, that Google’s AI ethics board is no more and looks to stay gone is not a positive sign. For any of their ML going forward.

That said, I have, now, both reread the paper and read many of the 107 cited works. Bottom line, as the paper itself says, it is but making a case for various research streams to coordinate for a specific outcome. And the hurdles between here and there are many.

Something to keep an eye on, research and patents, oh my, but not, yet, something to miss sleep over.

However, contingencies must be considered, for the trajectory is clear: as an info site I’m at risk.

[NickMNS]

Further, that Google’s AI ethics board is no more and looks to stay gone is not a positive sign.

Another side note to further point made @iamlost above. There is an excellent documentary called "Coded Bias" that reveal a few of the many ways in which algorithm can be biased.
[youtube.com...]

The full documentary is available on Netflix (at least in Canada), or it can be streamed for free from the PBS - Independent Lens website in the US.
[pbs.org...]

www.youtube.com YouTube is not currently available on this device. To learn more, please visit the YouTube Help Center:https://www.youtube.com/help

[Wilburforce]

you're dismissing what doesn't confirm to your pre-existing belief and rationalizing ideas (not facts) to shore up your belief

I'm sorry: perhaps you can tell me what my pre-existing belief is.

What I'm dismissing is drawing inferences from or basing arguments on incomplete data.The statements that "CTR to the organic web has steadily decreased over the years" and "We send billions of visits to websites every day, and the traffic we’ve sent to the open web has increased every year since Google Search was first created" are not mutually exclusive. If the number of pages - a statistic not referenced in either statement - has risen faster than the number of clicks*, overall CTR must have fallen. That isn't a belief, it is an arithmetic fact. Whether there are, have been, or could ever be zero-click searches (or whether, regardless of the answer, there might be a vaild reason for them) is not addressed by either statement or by any statement of mine.

I'm also disputing the suggestion that no Google change has ever reduced or will ever reduce CTR to any page. It is absurd to suggest that no sites or pages are ever adversely affected (whether in CTR or otherwise) by Google changes, as black-hat practices are intentionally so affected. Neither is it impossible - or even improbable - that a change targeting black-hat practices could affect an innocent page.

I'm not personally particularly anxious, but on behalf of anyone who is I also feel I should point out that fearing the outcome of a proposed change which will profoundly affect the way search operates is not stupid or irrational if, like most of us here, it will affect you directly and unpredictably.

_{* Does "the traffic we’ve sent to the open web" mean the same thing as "number of clicks"? I'm not sure, but it is also an arithmetic certainty that if the size of the open web has grown faster than "the amount of traffic we've sent", then sent traffic per web unit has reduced.}

[martinibuster]

...if Googlebot does it is "indexing" and its ok, but if another bot "indexes" your content and re-uses it on their site its spam. Got it.

No, you don't "got it."

Indexing is what happens when a company gives you the opportunity to block it. Scraping is what happens when a bot scrapes at will without giving you the opportunity to formally block it or even slow it down.

The technology described in this paper continues to push the limits and further erodes the agreement, essentially it pushes Google "indexing" ever closer to "scraping".

I agree with you there are ways of conceiving it that can be troubling. But the form that MUM and this new paradigm will resemble is not known, not even Google knows. This is fact and reality must be acknowledged.

The kinds of answers this approach solves are not addressed by current models served served by featured snippets, knowledge panels or 10 blue links. Whatever this new model looks like, with (or without) attribution to websites, I'll leave to your imagination. I have my own ideas.

I am certain that should this apocalypse where Google co-opts your content without attribution should finally (after over a decade of prognostication) come to pass, there will be a massive backlash.

But we are nowhere near that point, because we are at the fact pointed out above, we are months to years away from anything rolling out.

But to suggest that we should blindly trust Google, and that any technological innovation that is good for Google is good for us is just as nonsensical."

Nobody is suggesting we blindly trust Google.

The only suggestion I have made is to look at the facts and respond to that.

I am also asking everyone to not allow the conditioning of years of misleading clickbait articles about zero clicks and Google co-opting your content to influence your view of reality. Those are false facts that you've been exposed to, mainly coming from certain entities in this industry who have a history of concern for traffic over the veracity of their claims.

The hysteria is what happened when knowledge graphs were announced in 2012 and many at WebmasterWorld panicked under the false belief that the knowledge graph announcement heralded the end of the web as we know it. All I'm saying is, let's not do that again and be wrong yet again.

I am just saying, slow down, and look at the facts, uncolored by biases, so we can react in a proper manner without the hysteria of the past.

[paybacksa]

Not surprising that Google openly discussing it's perspective on the next big thing in the vertical it dominates (formerly known as "search"). I honestly don't expect Google to be much more than the next AOL closed community pretty soon, except with alot more market power. It hasn't been "search" since it stopped maintaining a primary index, supplemental index, etc.

Just because no other company stepped up to provide actual web search, doesn't mean a canned, pre-sorted, curated, and censored index of URLs is "search".

Not surprising that certain talking heads follow up with doom-and-gloom, sky-is-falling posts, and other certain talking heads follow up with condescending "you're an idiot" posts.

What is surprising, is that no one has yet commenting on how Search Engine Optimization (SEO) is going to need rebranding. The old "it's not actually 'optimizing search engines' so why do we call it SEO?" discussion is hereby dead.

[Wilburforce]

I am also asking everyone to not allow the conditioning of years of misleading clickbait articles

I actually find myself almost fully in agreement here (not just on that point, but on your whole post), but the more I think about it the less I think facts are the problem. What is stirring up consternation puts me in mind of an observation atttibuted to Aron Nimzowitsch: The threat is stronger than the execution.

It arose in a chess tournament in which his opponent took out a cigarette case, and Nimzowitsch complained to the umpire, who responded "He has not lit a cigarette and there is no smoke, so your complaint is noted but is not valid". Nimzowitsch answered "I know, but he threatens to smoke, and you know as well as I do that in chess the threat is often stronger than the execution".

The point is that it doesn't matter whether Google actually carries anything out. It is the threat that worries this community, not the executuion, which - as you correctly point out - has not previously come to anything like the devastation feared from earlier threats.

What has actually come to pass - the death by 1,000 cuts - is nevertheless enough to worry e.g Tim Berners-Lee, and I think it would be a mistake to view Google's actions or influence as benign. Google's primary duties - within the law - are to shareholders, not to us or the public at large. Why anyone would publish a paper saying "we're not ready to do anything yet, but we could make the sky fall one day" is debatable, but from what I see in this thread it has clearly succeeded in obfuscating much and revealing little.

[martinibuster]

"...in chess the threat is often stronger than the execution...

...from what I see in this thread it has clearly succeeded in obfuscating much and revealing little."

Nice points! Love it. ;)

I try my best to bring some clarity, as much as is possible, by writing about the algorithms in a way that the lay person might understand it, to cut through some of that obfuscation but much of the opacity of the execution remains (as well it should).

[ronin]

Increasingly, there ought to be enough space in the Text Search Vertical for both Google and Google Classic.

"Nature abhors a vacuum" - Aristotle (384 BC - 322 BC)

[JS_Harris]

The Q&As are full of wrong answers, often provided by 'trusted' Wall St listed mega sites offering a scraped answer for everything. Is Google really ready for a next more complex step before they fix that?

One particular wrong answer has irked me for several years because I know that if I write about it it will look like wrong information to Google and I may suffer in the rankings because of it.

Specifics: Company A was acquired in a process that took from 1951 to 1954 to complete. Company B then sold the brand and rights to company C in 1994. When you use keywords to search for company A you find the top 3 are fan sites with the right information. When you look at the FAQs and questions like 'What year did company A go out of business?' Google proclaims 1994, the same year the big answer sites proclaim. By 1994 company A had not existed on its own for 40 years...

In 2-3 keyword searches Google got the answer right, as evidenced by the top 3 results. In more complex question searches however... wrong, and now showing mega answer site answers instead of those 3 fan sites?

What I want from Google is links to the more specific smaller sites, not answers from big answer sites and certainly not Google giving me information I know they found on the net somewhere without letting me go verify.

[brotherhood of LAN]

The Q&As are full of wrong answers

Yes, I've found examples of this on a number of occasions.

The main point I'd make is that Google is entitled to do as they please, but given their market dominance - their answers can be akin to false information being spread on social media, and I don't know of anyone that denies that it's a problem. We're talking about a portal to the web that has 95% market share in some Western countries where search accounts for 50% of information discovery.

Being a portal to the rest of the web is what a 'traditional search engine' is - and fine, if times have changed then there needs to be room for someone taking the place that Google is leaving behind. Being able to answer queries with unequivocal truths like "what is the time in London" is one thing, but probabilistically generating an answer on unstructured data is the other end of the scale.

There's irrefutable evidence of Google's position and the ROI they make per search pricing other engines and potential new entrants out of the market. e.g. being the default search option on devices, browsers.

As long as there's room for an alternative point of view, a way to search past the headline answer - I can live with what Google may choose to do next.