Ah, if only that were true in the real world. ;-) Unfortunately that sentiment reminds me of SE's telling everyone to just do what's best for users. Puhleeease. Hehe.
The problem is, the algos still going strong today put a very, very heavy emphasis on links. So much so that, as the war on spam intensified several years ago, lots of very high quality mom and pop sites got blasted out of the SERP's for lack of links.
And the problem is currently worse than ever, because most site owners are so stingy with doling out those little gems. In today's world, you can't reciproate, can't beg, and can't buy. Man o man o man...what's a little site to do, really? Keep working like crazy in hopes that the world will notice? Just because you build it, that doesn't mean they will come.
Strip wikipedia and about 10 other mega sites out of the SERP's and the SERP's would look ENTIRELY different. Huge sites with massive link-based authority dominate today's search landscape.
Wish it wasn't so, but personally, I try and deal with the real world, rather than a hopeful reality that only I wish was so. :P