Welcome to WebmasterWorld Guest from 3.93.74.227

Forum Moderators: goodroi

To Break Google’s Monopoly on Search, Make Its Index Public

from Bloomberg News.. do you agree?

     
11:50 pm on Jul 15, 2019 (gmt 0)

Preferred Member

10+ Year Member Top Contributors Of The Month

joined:Apr 13, 2004
posts: 497
votes: 88


[bloomberg.com ]

I like this idea..

And not only make the index public, allow websites to exchange links with each other again without penalty, the way the web was designed to work!

Thoughts? Should the search index be public?

[edited by: goodroi at 12:04 am (utc) on Jul 23, 2019]
[edit reason] fixed url [/edit]

5:48 am on July 24, 2019 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:10586
votes: 1125


Gaining traction in high places over the last few days.

Will have to see how this shakes out.
5:30 am on July 25, 2019 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Aug 5, 2009
posts:1731
votes: 388


The only idea that works is having a not for profit search engine. No ulterior motives. Wasn't Cutts trying to get into this field? What a shame all of his ability cannot be used to better the planet. Bettering the planet by allowing people to use the internet in a way that isn't about filtering or using for the sake of profit. How about finding websites that are good because there is honor in that. The solution is to allow Cutts to create a true not for profit search engine. A not for profit search engine that might actually be good is a threat to Google profits so how could they let him use what he knows? He knows too much. Imagine learning a level of expertise that is so tied into a company so that when you leave, you become a vegetable. You can't access and use all that you know. In terms of the linked article? I have two words. Say what?
7:28 am on July 25, 2019 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Nov 13, 2016
posts:1194
votes: 288


For a while Wikipedia was supposed to develop a search engine... (yes I know some will say "don't tell me about these #### of Wikipedia").

I agree that Matt should be the one leading a real free independent search engine project, but comes the question of financing. When you look at big free things, you end with Google, Microsoft, Facebook etc... as business "angels", which then can be a problem.

If no one does it, I'll need to do it by myself :)
8:19 am on July 25, 2019 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:10586
votes: 1125


If no one does it, I'll need to do it by myself :)


Post the roll-out date! I'll join for a chuckle --- or wowser!
4:23 pm on July 25, 2019 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Nov 13, 2016
posts:1194
votes: 288


The harder will be to find another nickname, for not being backfired by some of my trolling.
5:32 pm on July 25, 2019 (gmt 0)

Preferred Member

10+ Year Member Top Contributors Of The Month

joined:Apr 13, 2004
posts: 497
votes: 88


But Cutts is the very author that published the "you can't link exchange with like minded sites without being penalized" crap ..so how can he be pro public search and anti linking at the same time?
5:52 pm on July 25, 2019 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Aug 5, 2009
posts:1731
votes: 388


Cutts is the closest thing you will find to ethical. The bit about can't link exchange? I would rather look at his stance/comments about paying websites/content creators for times when Google uses that content in their answer boxes. Because Cutts didn't run Google, he took directions from other people. As far as moral compass? Doesn't the fact he doesn't work at Google say something? He essentially gave it all up. His existence, expertise evaporated over that decision. Not a small decision in my mind. It's one of those decisions that says, morals or livelihood. He chose morals. At least that is what I want to believe.

In terms of financing? Yes, there is always that. People don't trust government, but I trust governement(s) more than profit seeking corporations. There needs to be a situation where there is no motive to filter traffic. It has to be simply about finding the best website for what you're typing into the box. No profiling required! Google search worked just fine before they decided to take your data and use that into their "system".

A joint effort on a search engine, government driven with an ethical person at the helm. That's my suggestion.
7:19 pm on July 25, 2019 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:July 29, 2007
posts:2014
votes: 215


Not enough.
Recognition is growing worldwide that something big needs to be done about Big Tech, and fast.

Yeah, they need to stop being evil and trying to influence elections(in any direction) among other things. Sorry, but they need to be dismantled because they've proven that they will silence people.

Google has placed Greg Coppola, a senior software engineer, on administrative leave for telling the truth about Google’s far-left bias in its search engine results. Not even their own employees are safe, Google has gone politically mad, it's no longer "just a search engine". Supporting the left(or the right) is fine but silence anyone and now we have a serious problem from a search engine and advertising company.

Break it up, I personally already commited to avoiding any and all Google services possible. Adsense is all I have left from G. I don't trust them and it's not so much corporate practice as the very extreme left mindset of it's "leaders" and that they think they are needing to silence anyone who disagrees.

added: "but we have to fight fake news...". No, you used to fight fake news as it struggled to rank but now you say that when you silence news you don't want people to see or hear about. Example: Just today a video was released by police of 17 colored attackers swarming and severely injuring an elderly white guy and his wife who were walking into a D..C. Hilton hotel. The left won't allow it to receive attention, but it's not fake news. It's unfriendly to the left's narratives but it definitely happened. Nothing will change if you can't even talk about it anymore.

Social media too - search the word Trump and you can see severed head tweets but search the meme "derp squad" and anyone who retweets it is banned. This isn't fighting fake news, it's being very biased. Big tech got political, fine, now their platforms need to be regulated so as to not have such biases be possible. Report it all, allow everyone to speak, or implement a no politics policy equally on everyone. Big tech has failed spectacularly of late.

Cutts is the closest thing you will find to ethical.
We were lucky to have him as a guidance for Google's policies, they've struggled more since he left.
8:35 pm on July 25, 2019 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Apr 1, 2016
posts:2740
votes: 839


And not only make the index public, allow websites to exchange links with each other again without penalty, the way the web was designed to work!

This statement clearly shows that you have no idea about what you are talking about. The index, should actually be called the database. I fully agree that it should be public, the collected data are technically not the property of Google. The data essentially consisting of copies of websites are in essence the property of the site owners.

allow websites to exchange links with each other again without penalty

You are free to link to what ever website you like. Google's ranking algorithm may penalize you as a result but your are still free to do it.

This brings up the second point, the ranking algorithm, Page Rank et al. is not the part of the index. Make the index public and then allow anyone else to develop a ranking algorithm to suites their needs. You could then have search engines specialized in finding cat images, others for link farm websites, whatever. Their could then easily be opensource algorithms or full search engines developed.

Not only do I think it is a great idea, I think it is the only way to be able to break Google's grip on the web.

In the same manner I think that social media content should also be free and open. Users could then decide on which platforms their content would appear and be able to add or delete content at will, as well as switch networks seamlessly. FB would then be open to actual competition and would need to work on providing top notch user interface platform, search and data feed, and not simply work on walling in your own data and acting as a gate keeper.
10:02 pm on July 25, 2019 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:10586
votes: 1125


Kiddies, I also love unicorns and fairy tales, but both do not exist, and it is quite likely that g, fb, amazon, and twit will fight tooth and nail to protect their patents, trademarks, et al.

What DOES exist that can bring back fair play is REMOVING Section 230 and then HOLDING these tech giants to the same standard as EVERY OTHER PUBLISHER, BROADCASTER, and MEDIA company. With that in place (and possibly breaking up the mega corps for anti-trust infractions) the web will have a chance to heal AND reveal, any bias currently hidden.

A general purpose index would be nice, but as NickMNS points out, it is data. HOW it is structured is each company's patented/trademarked bread and butter. That said, who will reimburse G for their investment is obtaining all that data? Will others have to BUY it just to "compete"?

There are no simple answers, but there are many HARD QUESTIONS.
2:04 pm on July 29, 2019 (gmt 0)

New User

5+ Year Member

joined:Oct 31, 2010
posts: 22
votes: 4


Yes google will fight tooth and nail to protect their patents, trademarks, copyrights etc and above all their revenue stream so they wont make the index public unless they have to and I believe they will have to at some point, and...…….the sooner the better.

I mean what right do they hold over information they've scraped from readily available and publicly accessible web sites, information & data that they didn't even create themselves? None. The only argument they have is that they have spent money and resources into obtaining that information, yea they've also MADE money, a lot of money....from that scraped data.

I've read the following on a few news articles about the matter:

"There is precedent for this both in law and in Google’s business practices. When private ownership of essential resources and services—water, electricity, telecommunications, and so on—no longer serves the public interest, governments often step in to control them."

Making the index public would be brilliant, I mean how many other search engines would be born? It would give the user a wider range of choice regarding internet search, bring about healthy competition and lead to more innovation in web search. Let them keep there precious algorithm under lock and key, who cares.

A better search engine would be born sooner or later, guaranteed. Or Google themselves would remain dominant but become a better search engine as they would no longer be able continue focusing on squeezing every last penny from ads while taking it away from publishers.

They really would have to concentrate and focus on providing the user with good, relevant results rather than providing the user with results that make them the most money from publishers who pay for traffic.

It would also stop the iron grip and monopoly they have over web search, where lets face it, they hold publishers to ransom basically...….its becoming either pay them for traffic or were gonna cut off all decent traffic you get from us,,,period.

If they really wanted to 'support the ecosystem' as danny sullivan said, what better way than making the index public? They should have nothing to fear anyway right? They are number one in internet search, they have been developing their algorithm now for over 20 years, surely they provide the best and most relevant results to users? *cough cough*. I mean the competition could'nt even come close right? lol after all their main goal is to support the ecosystem so why not? :)

Also, this new development with longer snippets in SERPS. Well a page on my site that used to convert very well and held position 5/6 consistently has recently moved to number 3, thats good right? Erm not so much, it now has a massive snippet of bits of the page in different areas of the description and reads awful.

CTR and conversion have decreased....a lot. Longer snippets is just another method used to stop users clicking on the organic results and to get them clicking on the ads instead.

So result number 3 which appears a loooooooong way down the page after all the ads and the ever so helpful and user oriented search features of course but now with a massive snippet of random sentences plucked from various parts of the page that really should be an example somewhere of how NOT to write a meta description if you want to maximise CTR.

Google have become far too greedy and it will be their downfall eventually.
4:37 pm on July 29, 2019 (gmt 0)

Senior Member

WebmasterWorld Senior Member topr8 is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 19, 2002
posts:3513
votes: 87


this is all irrelevant to me.

even if somehow there was a universal search index/db available to all (eg. currently google's) ... i don't understand how anyone else would make a 'better' search engine.

after all, they all would be grabbing as much data as possible from users and funding/profiting their engine with ads and/or selling data (to a greater or lesser extent) ... just like the current lot do. why would someone else's ad riddled results pages be better than google's?

as for the idea of a so called non-profit search engine, let's think about that for a minute.
imagine there was one (using this universal magical database)
imagine it didn't run ads.
imagine it didn't collect data.
... well who would pay for this utopian search engine?
someone somewhere would, either a government or a deep pocketed individual or a rich charity.
do you really think any of them wouldn't have an agenda?
like: oh we don't like left/right wingers - lets push them down the serps.
we don't like goods made in china/usa/europe/africa - lets push them down the serps.
actually we don't like ecommerce sites - lets push them down the serps.
hmm, how about religions/beliefs ... we don't like x/y/z religion or infact any of them at all, lets hide them in the serps.
... all for the greater good of the users of course.

i also agree with tangor

What DOES exist that can bring back fair play is REMOVING Section 230 and then HOLDING these tech giants to the same standard as EVERY OTHER PUBLISHER, BROADCASTER, and MEDIA company.


i'm not from the usa so i'm not sure what section 230 is in detail, but i assume it is a statute that somehow removes liability away from various internet platforms.

potentially there could also be introduced a system whereby spiders have to pay to spider websites (eg. the big search companies would have to pay to collect the web data they use/scrape) - however wait for the complaits when site a/b/c is not spidered as the search giant figures they are not worthwhile at all - eg. you're not in the index at all.
2:18 pm on July 30, 2019 (gmt 0)

New User

5+ Year Member

joined:Oct 31, 2010
posts: 22
votes: 4


even if somehow there was a universal search index/db available to all (eg. currently google's) ... i don't understand how anyone else would make a 'better' search engine.

after all, they all would be grabbing as much data as possible from users and funding/profiting their engine with ads and/or selling data (to a greater or lesser extent) ... just like the current lot do. why would someone else's ad riddled results pages be better than google's?


Your presuming that all search eninges created would be as ad riddled and focused as google has become. Theres a difference between displaying ads alongside organic results to genreate revenue and pushing organic listings down as much as possible so that organic CTR's decline and paid ad CTR's increase.

According to this article [searchengineland.com...] since 2016 clicks on organic results have declined from 45% to 36.7% and clicks on ads have increased and more than doubled in the UK-EU zone while in the U.S clicks on organic links have declined from 40.1% to 29.7% and paid clicks on ads have more than doubled.

Google are basically putting content in front of people from publishers who pay them the most money and demoting all other content as much as they can get away with. Its only going to get worse with time if something isnt done about it.

as for the idea of a so called non-profit search engine, let's think about that for a minute.
imagine there was one (using this universal magical database)


I dont think there will ever be a non-profit search engine as serving thousands or millions of users costs money but at least if there was a wide variety of different options then users could decided for themselves which search service they prefer regardles of whether they show ads or their biases or whatever.

At the moment google is gatekeeper to this information and is basically selling and promoting it to the highest bidder.

I would imagine a lot of niche search engines would emerge that concentrate and specialise in particular subjects and do it a lot better and a lot fairer than google ever could or will.
4:48 pm on July 30, 2019 (gmt 0)

Preferred Member

10+ Year Member Top Contributors Of The Month

joined:Apr 13, 2004
posts: 497
votes: 88


"At the moment google is gatekeeper to this information and is basically selling and promoting it to the highest bidder. "

and breaking dozens of Federal Laws in the process.. the entire board at G should go to prison.
6:06 pm on Aug 1, 2019 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Aug 30, 2002
posts:2663
votes: 109


For a while Wikipedia was supposed to develop a search engine... (yes I know some will say "don't tell me about these #### of Wikipedia").
No. It was a plan by Jimbo and some others to get people to build a search engine for free. It didn't end well because the people with a clue weren't going to do that and one of them even asked "why should we make you rich?". :) The biggest flaw was that none of them understood the importance of a clean and current index.

Regards...jmcc
10:20 pm on Aug 1, 2019 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Aug 30, 2002
posts:2663
votes: 109


The term "not for profit search engine" is ruefully funny to those of us who have ever built a search engine. Short of making a search engine pay for inclusion, all search engines consume time, resources and money. It is advertising and placement that make the difference. The only reason that Google managed to grow is because it successfully monetised Search. That's the biggest problem, after a clean index, that a search engine has to overcome.

API access to Google's index is an interesting idea but Google would probably use the data from such accesses to build their own vertical search engine if it was successful. Basically Google has a history of plagiarism. It tried to do it with Facebook and failed. It tried with Twitter. The problem is that with these projects it had no entrepreneurial spirit of the type necessary to set up and run such a new venture.

Even with API access to Google's index, it comes down to one single question: Do you trust Google?

Regards...jmcc