I'm really curious to find out where people in thies forum stand on the issue of PFI. Let's take a quick vote. There seems to be 3 main view points when it comes to PFI:
1. AGAINST PFI - It's deceptive, gives unfair advantage to those that pay, and hurts overall relevancy.
2. NEITHER FOR NOR AGAINST - It does not impact the results one way or the other
3. FOR PFI - It helps make search results better by allowing otherwise non-indexable content to be indexed
I myself vote for option 1 - AGAINST PFI.
In my opinion, anyone who fully understands how PFI works (and is honest with themselves) has to admit that there are serious, fundamental flaws with the PFI model. I have worked very closely with many of the top search engines in the past to develop these programs, and have come to the conclusion that PFI is a fatally flawed model that is destined for extinction.
Here's my take on it: The end goal of a search engine is to build a thorough database of as many pages as they can, and to continually add more fresh content to keep it up to date. The goal of PFI is to make as much money as possible from people paying to have their sites "included" in that index.
I believe there is a serious, fundamental conflict of interest with these two end goals. Both cannot flourish at once. The more one goal is met, the more difficult the other goal becomes to acheive.
If a search engine was truly trying to build the best possible index, it would promptly and frequently crawl all urls for free, in order to keep their index fresh and complete. If they did that, however, there would be absolutely no incentive for anyone to pay to have their URL's included. PFI creates an incentive for the search engine to NOT spider the entire web... and to NOT continually add more and more fresh good content.
A long time ago, Inktomi was faced with an intersting dillemma (i was part of this discussion)... How do they spider the web for new content, but still make people feel the need to pay to be included. They dealt with this by spinning it such that you were paying to ensure "fast and frequent" spidering. In other words, the only way they can make it seem worhtwhile to pay, is to admit that their search engine is poor quality since it cannot go out and index the web quickly and often by itself.
(If they're smart, Yahoo will now recognize that Google is doing just that... they're indexing more and more of the web, and they're keeping it fresher and fresher over time. MSN is about to come out with their own spider and index soon... with Bill Gates vowing revenge on Google, do you think Microsoft has the resources to build a search engine that can spider and index the web and keep it fresh and up to date? (Hint: $50 Billion in cash).
Another main conflict of interest that Inktomi discovered a long time ago: What to do when someone with a 500 page website, full of good fresh content, submits just a few pages of the site? Well... if they were TRYING to build the best possible index, they would go ahead and crawl the rest of that site for free, to be sure they included that good content in their index for people to find. Of course... if they did that... they realized that this person would never again pay for another page of their site to be included!
So what did they decided to do? They decide to NOT index the rest of that good content... and rather decide to ignore it in the hopes that they can make a few extra bucks by forcing the webmaster to pay to have those pages included as well. This is a great example of choosing short term money over the best interest of their search engine users.
One more example of what makes PFI such a terrible model: What to do when someone who ALREADY has 500 pages of content in the Inktomi dbase, but submits one of their pages. Yer gonna love this one... they decide that since you've proven your willingness to pay for inclusion, we'll just go ahead and pull the other 499 pages from our index... in the hopes that you'll pay for all those too. This is the most dramatic example of how willing Inktomi is to sacrifice good quality content in order to make a fast buck. I know for a fact this was their policy up until awhile ago... and i have no reason to believe this policy has changed.
This is not to mention the issue of "Trusted Feeds" that allow PFI firms the ability to create cloaked doorway pages and manipulate the search results such that their feeds rank higher. Even proponents of these types of feeds in this forum admit that these "Trusted Feeds" can acheive higher ranks if the company preparing them optimizes them correctly. How is that fair?
Final point: if the average user knew that the results in the main body of MSN were influenced by money, they would be surprised. I always ask friends and family and they always seem surprised to hear that. Ralph Nader even filed a complaint claiming PFI was "deceptive advertising". PFI operates in the shadows. The engines know that its a concept that few people understand enough to be appalled by. However, the more savvy people get as time goes on, the more offended they are at the concept.
It may sound like i am bashing Inktomi/Yahoo. I am not. In fact, I very much applaud Yahoo for the recent changes and congratulate them on a great new index. It's merely my desire to see a healthy variety of high quality search engines, competing with each other for the best product, that makes me so against the concept of PFI. Yahoo simply CANNOT compete with Google in the long run with a PFI-based search engine that lacks the type of fresh, thorough content that Google strives for month after month (even if they are a bit goofed up right now :)
So there's my explanation of why i vote #1 - Against PFI.
What's your vote?
[edited by: TrafficL at 6:34 am (utc) on Feb. 20, 2004]
