Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

Cast Your Vote on PFI!

Pay for Inclusion --- Good or Evil.



6:02 am on Feb 20, 2004 (gmt 0)

10+ Year Member

I'm really curious to find out where people in thies forum stand on the issue of PFI. Let's take a quick vote.

There seems to be 3 main view points when it comes to PFI:

1. AGAINST PFI - It's deceptive, gives unfair advantage to those that pay, and hurts overall relevancy.

2. NEITHER FOR NOR AGAINST - It does not impact the results one way or the other

3. FOR PFI - It helps make search results better by allowing otherwise non-indexable content to be indexed

I myself vote for option 1 - AGAINST PFI.

In my opinion, anyone who fully understands how PFI works (and is honest with themselves) has to admit that there are serious, fundamental flaws with the PFI model. I have worked very closely with many of the top search engines in the past to develop these programs, and have come to the conclusion that PFI is a fatally flawed model that is destined for extinction.

Here's my take on it: The end goal of a search engine is to build a thorough database of as many pages as they can, and to continually add more fresh content to keep it up to date. The goal of PFI is to make as much money as possible from people paying to have their sites "included" in that index.

I believe there is a serious, fundamental conflict of interest with these two end goals. Both cannot flourish at once. The more one goal is met, the more difficult the other goal becomes to acheive.

If a search engine was truly trying to build the best possible index, it would promptly and frequently crawl all urls for free, in order to keep their index fresh and complete. If they did that, however, there would be absolutely no incentive for anyone to pay to have their URL's included. PFI creates an incentive for the search engine to NOT spider the entire web... and to NOT continually add more and more fresh good content.

A long time ago, Inktomi was faced with an intersting dillemma (i was part of this discussion)... How do they spider the web for new content, but still make people feel the need to pay to be included. They dealt with this by spinning it such that you were paying to ensure "fast and frequent" spidering. In other words, the only way they can make it seem worhtwhile to pay, is to admit that their search engine is poor quality since it cannot go out and index the web quickly and often by itself.

(If they're smart, Yahoo will now recognize that Google is doing just that... they're indexing more and more of the web, and they're keeping it fresher and fresher over time. MSN is about to come out with their own spider and index soon... with Bill Gates vowing revenge on Google, do you think Microsoft has the resources to build a search engine that can spider and index the web and keep it fresh and up to date? (Hint: $50 Billion in cash).

Another main conflict of interest that Inktomi discovered a long time ago: What to do when someone with a 500 page website, full of good fresh content, submits just a few pages of the site? Well... if they were TRYING to build the best possible index, they would go ahead and crawl the rest of that site for free, to be sure they included that good content in their index for people to find. Of course... if they did that... they realized that this person would never again pay for another page of their site to be included!

So what did they decided to do? They decide to NOT index the rest of that good content... and rather decide to ignore it in the hopes that they can make a few extra bucks by forcing the webmaster to pay to have those pages included as well. This is a great example of choosing short term money over the best interest of their search engine users.

One more example of what makes PFI such a terrible model: What to do when someone who ALREADY has 500 pages of content in the Inktomi dbase, but submits one of their pages. Yer gonna love this one... they decide that since you've proven your willingness to pay for inclusion, we'll just go ahead and pull the other 499 pages from our index... in the hopes that you'll pay for all those too. This is the most dramatic example of how willing Inktomi is to sacrifice good quality content in order to make a fast buck. I know for a fact this was their policy up until awhile ago... and i have no reason to believe this policy has changed.

This is not to mention the issue of "Trusted Feeds" that allow PFI firms the ability to create cloaked doorway pages and manipulate the search results such that their feeds rank higher. Even proponents of these types of feeds in this forum admit that these "Trusted Feeds" can acheive higher ranks if the company preparing them optimizes them correctly. How is that fair?

Final point: if the average user knew that the results in the main body of MSN were influenced by money, they would be surprised. I always ask friends and family and they always seem surprised to hear that. Ralph Nader even filed a complaint claiming PFI was "deceptive advertising". PFI operates in the shadows. The engines know that its a concept that few people understand enough to be appalled by. However, the more savvy people get as time goes on, the more offended they are at the concept.

It may sound like i am bashing Inktomi/Yahoo. I am not. In fact, I very much applaud Yahoo for the recent changes and congratulate them on a great new index. It's merely my desire to see a healthy variety of high quality search engines, competing with each other for the best product, that makes me so against the concept of PFI. Yahoo simply CANNOT compete with Google in the long run with a PFI-based search engine that lacks the type of fresh, thorough content that Google strives for month after month (even if they are a bit goofed up right now :)

So there's my explanation of why i vote #1 - Against PFI.

What's your vote?

[edited by: TrafficL at 6:34 am (utc) on Feb. 20, 2004]


6:07 am on Feb 20, 2004 (gmt 0)

10+ Year Member

Great post. I won't vote because I'm using PFI and am OK with it, but your points are so full of dead-on, eye-opening logic, that I agree with the negative aspect completely.


6:15 am on Feb 20, 2004 (gmt 0)

10+ Year Member

I would stop far short of calling it evil. Like the old saying goes, business not personal. Each engine has a right to do their own thing. That said, if you look at the financial fate of search companies, PFI does not appear consistent with maximizing shareholder value in the long run.


8:31 am on Feb 20, 2004 (gmt 0)

10+ Year Member

For obvious reasons I am not going to voice my opinon.

However, I feel there has been a key thing missed here - what about the search engines need to make money to pay its expenses.

Search engines are not a public service.
They cost a lot of money to run - this revenue needs to come from somewhere.

In this world, there isn't too much for free - it is very much a user pays system. The user - in this case - is the advertiser.

Perhaps the search engines should start charging the "real" user - the one doing the searching instead of the person doing the advertising?



1:50 pm on Feb 20, 2004 (gmt 0)

10+ Year Member

Good point Warren. However, Google made $700 million the first year Adwords was introduced. No one knows their exact financials, but most predict they have a VERY healthy profit margin.

Adwords are clearly separated from the relevance matches and are clearly marked as sponsored listings. Google does not use PFI.

I'd say Google has proven a search engine can do quite well without needing to foul up their relevance listings with PFI.


2:18 pm on Feb 20, 2004 (gmt 0)

10+ Year Member

Against PFI!

1) Since only those that can afford it get all pages indexed, all pages are not available to the search engine using it. Means the SE will not have all the information and therefore cannot be the best.

2) Even at $25 per URL, NOBODY can justify PFI for a 20,000 page site! And don't tell me that all pages will get indexed and included if you just pay for the index page -- does not happen.


2:49 pm on Feb 20, 2004 (gmt 0)

WebmasterWorld Senior Member pageoneresults is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

Unfortunately the bottom line is...

If you want traffic from those resources, PFI is your best option. Many of us realize that the PFI model has its flaws but there really is no way around it unless you are willing to sit and wait for Slurp to come by and start indexing your site. That could be quite a long wait and you may starve before then. ;)

I'm for PFI when applicable. If you want MSN, AV and AJ traffic, then PFI is probably the best alternative.

The ROI on PFI pages is usually worth it. At $25.00 per year per URI, you can recoup your investment usually within 30 days. In many cases within the first 48-72 hours after going live.


3:02 pm on Feb 20, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

Against PFI.

Infact i'm considering starting an anti-pfi domain and creating logos and images for webmasters to use to push.

I am also going to write up documents on how to block PFI engines from spidering your site.

After all even if you get free inclusion it is THEM profiting from YOUR content and webmasters have the authority to decide how and when you can find there content.

I will als happily link to search engines (google) that don't use PFI.

PFI isn't affordable, PFI isn't manageable, PFI isn't ethical and PFI doesn't make sense.


3:26 pm on Feb 20, 2004 (gmt 0)

10+ Year Member


Correct. Adwords is a great product and a good revenue spinner for Google.

But if we step back and look at Inktomi prior to being a Yahoo! company, they needed revenue to operate. They don't have a Paid Placement product that they could make money from.

PFI - like it or not - is something that is here to stay because it does allow companies such as Inktomi and Teoma (and of course, lets not forget AV and AlltheWeb/Fast) to have revenue and continue to have a site and offer a service to the public.

At the end of the day, companies need to pay people's wages. PFI allows this to happen.




3:40 pm on Feb 20, 2004 (gmt 0)

10+ Year Member

Warren - Yes, I agree. Prior to Yahoo's purchase of Inktomi, Ink needed a better revenue model. PFI is one way for a search engine to make money.

... and I also agree with Pageonresults... ADVERTISERS CAN make good money with PFI as well.

The only other party left out of the equation is the end user of the search engine. What is best for them? I believe that Google proves that engines can have their cake and eat it too - provide a great, fresh, up to date index AND make A LOT of money at the same time.

Now that Ink is part of Yahoo, with all the cash and resources Yahoo has at their disposal, PFI is not a necessity for them to survive. It's just short-sided greed for more. In the long run, I believe engines will make much MORE using clean results that people trust... when the user trust erodes and no one uses the engine any more, what good will PFI be for the engine at that point?


3:59 pm on Feb 20, 2004 (gmt 0)

10+ Year Member


You are right. The end user does need to be taken into account.

Perhaps a model where the user pays for search could be done. This could be done using Microsoft's Passport/Hailstorm (or the Sun standard which name currently escapes me) services which also include a high level of customization for the user.

A true user pays model.

Whilst I do believe that Passport will play a role in MSN Search moving forward, I don't think it will be used to bill the user. More to provide a customized experience.

I will point out - in order to appear as neutral as possible - that I work for a PFI reseller and was formly employed by Microsoft Australia. Hence my views may be slighly tainted on this matter.



6:31 pm on Feb 20, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

My vote: I am in favor of whatever the SE is in favor of.

TrafficL ~ your insight into to contradictions and difficulties within PFI is certainly appreciated and noble. It is about time that something struck a chord with you on WW. We can’t learn from you if you don't talk…. but you already knew that.

I will not take aim at your PFI concerns, as they are legitimate. Instead, I will take aim at what you fail to address: The fact that no Algo is without bias or manipulation, PFI or not.

Put another way, the pretty picture you paint for G fails to consider the fact that *you* manipulate an Algo based upon what you know of its bias.

So, on one hand you promote free (index every page in the world) serps for the good of the people, all the while, you (like us) are manipulating those same pages for the best rank. Not because we are more relevant, but because we know how to optimize our pages in accordance with the bias of the Algo. Money manipulates, but so too does knowledge.

We know that more than half of all users don't go to page 2 of serps, and we know that we are in a struggle for a finite amount of extremely valuable real estate. To fight the PFI battle for the people, when you or I are simultaneously competing against these same people is confusing logic, which makes me wonder how much you’re worrying about clicks on those thousands of pages. Maybe trusted feed will help.

PFI has flaws. Algos have flaws. And no serps are without strong bias.

Thanks for sharing your views, but Y! has made its decision and in the final analysis they are no different than you or G or I, we are trying to maximize our utility.


6:36 pm on Feb 20, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

Look, whether you manipulate serps to drive clicks on adwords or simply put ads in the serps, what is the difference, eventually Joe Surfer will decide the fate of the engine. In either scenario, if you turn the knobs too far you have problems.


8:18 pm on Feb 20, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

"only other party left out of the equation is the end user of the search engine. What is best for them? "

"What is best for them?"

Come on...Deep down we all have the same answer: "For them to find MY site"


8:18 pm on Feb 20, 2004 (gmt 0)

WebmasterWorld Administrator skibum is a WebmasterWorld Top Contributor of All Time 10+ Year Member

If people weren't in such a hurry to show "ROI", PFI wouldn't even be an issue. If nobody paid for it, the engine would still have to crawl the web if they wanted to have a search engine. Its a prisoners dilemma.

If there are two competitors and neither one used PFI or feeds they would both get traffic and after a while probably have many pages in the index for free.

If one of them pays and consistently ranks above the other one because they can manipulate the contents of the data feed, then the other one must pay to catch up. In the end they may still be on par with each other but they are both paying more to generate the sales they were getting for free before. To the bean counter, however, it looks great. We're spending X dollars on this program and getting XXX back. PSSSSSSSSST! yea but you weren't paying anything for it before, it just wasn't quantified!

I think PFI, Trusted Feed especially, skews the SE results and favors those throwing boatloads of cash at the engines (why else would advertisers do it?). Does it distort the search results, well in Ink over the last couple years, IMO, yes. Don't think we've ever had the problem of ranking to well in Google editorial results but with a feed, sometimes the rankings can really surprise you.

How will the new Yahoo! deal with it? That has yet to be seen. If its like Inktomi is today, then it could use some improvement. If Yahoo! can spin it so people buy it and still mainatain a fresh index by crawling the web regularly then so be it. That's what marketing is all about isn't it? Its convincing more people to buy more stuff more often whether they need it or not. Anybody here with a commercial website not in that game?

There has got to be a middle ground between the Ink of today and the Google of today. It seems G is going after pure volume of pages indexed w/o regard in at least some segments, to quality. Ink seems to be more focused on pages from advertisers at the expense of breadth, freshness & quality of the overall index. Maybe the new Y! index can strike a happy medium. Only time will tell.

PFI - Evil, but every advertiser out there makes the decision as to whether or not it will be a successful business model for Ink, Y, Jeeves or whoever is offering it that day. Not sure the engines should take all the heat for it.

Disclosure about what is paid and what isn't, now that's a whole different ballgame.


9:39 pm on Feb 20, 2004 (gmt 0)

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

Having been a consultant for a "victim" of forced PFI after an important search provider arbitrarily dropped a client's pages, I'm against it. This was a small bricks-and-mortar, mom-and-pop operation, and they did not have the kind of money needed to pay for all of their pages to be re-included; it was a small, non-e-commerce site. The same goes for expensive directory listings of dubious value -- $299 is nothing to IBM, but a lot to a small business during a recession.

I'm not a conspiracy-nut and this is not a rant, but I did take the dropping of their site, which was clean, useful, and "honestly-optimized" for local visibility, to be a form of blackmail.

By comparison, mom and pop were (and are) much happier with the PPC model available from other providers for the few keyword phrases where their pages were swamped by heavy competition.

Personally, I believe that PPC, presented as "Paid Advertising" with full disclosure is a much more honest approach. It is also manageable for advertisers and profitable for providers. A Pay-Per-Search model is doomed as long as there is one college student left on earth who is willing to code up a search robot and user interface to provide free search -- Ask my daughter if she wants to spend her weekly allowance searching... I don't think so. She's also a minor, and therefore we have the added child-privacy-protection issues with requiring an "account" for search, as well as the fact that daddy will not simply hand her a credit card to use at will.

If no college student can be found, the alternative will be a quick return to the "Jerry's Guide to the Web" model*, and pay-per-search will be relegated to corporate use. There are many ways to stave off the complete and utter commercialization of the Web; Jerry's model could still work, given just a few big directories and a few lower-level specialty directories -- As long as the users don't have to drill down too far, the extra convenience of search over directories may no longer be compelling enough if people are forced to pay for search.

Pay-For-Inclusion - Thumbs down.
Pay-For-Search - BIG thumbs down.


* Yahoo! co-founder Jerry Yang's original Web index.


12:24 am on Feb 22, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

I am definitely anti-PFI, as it exists now, after my most recent experiences. It's one thing when it actually works as advertised, but I have yet to see one which provides the benefits promised. Of all the domains I manage, the ones which HAVE NOT been PFId rank far better than those which have been, on the same engines. They've ALL apparently forgotten who is their lifeblood. Like every other business, they need to think long and hard: who can I really survive without? [a few top paying advertisers with a very narrow focus] or [wide and varied websites' content and the end-users/searchers who are looking for it] and THEN decide who they need to appease most!
Great Post SkiBum. BTW, specifically what types (brands) of "trusted feeds" are you referring to when talking about wonderously benefitting ranking if you don't mind sharing.


10:21 am on Feb 23, 2004 (gmt 0)

10+ Year Member

I'm pro PFI - I can get listed the day I put up a new site for a tiny cost.


10:40 am on Feb 23, 2004 (gmt 0)

10+ Year Member

I don't mind PFI because the company needs money to survive & thrive (though I think completely non-commercial sites should be given free listings... you know, the 3 that are left out there :P).

$40 PER page is insane though. I'd rather pay a flat reasonable fee to get all the pages on my site crawled weekly or even monthly rather than a couple of pages checked daily.


4:07 am on Feb 24, 2004 (gmt 0)

no way can Y! leave tons of sites out while G is adding billions more and still be competitive. Yahoo already has a PFI, it's called the Y! directory.

Add all the sites you can and let the chips fall where they may. Let their algo and users decide which one is worth a visit. $40 a page is a joke. Many of us have sites with 1000's of pages. That's at least $40k a year (if you pay), while Google indexes them for free. If it was $200-$300 a site (up to x amount of pages) it's different. I think in the long run they make more by free indexing and selling Overture ads because you get more acurate searches, therefore more users. How did Google become a verb?

I know it's tempting to get that $40 a page but I think the money is on the ads and selling other Yahoo services while they're searching there. This is just my opinion, I could be wrong.


3:38 am on Mar 7, 2004 (gmt 0)

10+ Year Member

Thanks everyone for the feedback.


3:47 am on Mar 7, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

Great if it serve's its purpose. Fast inclusion, reguarly refreshed content. I ain't yet seen a PFI model with whatever claimed re-index refresh rate come up on it, even when url's have been changed in their programs.

The PFI's that i have experienced over the last couple of years, great for the inclusion - ashame about the rest of the commitment on the refresh rates.


5:10 am on Mar 7, 2004 (gmt 0)

10+ Year Member

After paying registereverywhere to list me with all the search engines and learning how to optimize my site with title, description, meta tags, etc....
I had my website ranking 1,2,3,5,& 8 with my major search terms and then I signed up with Overture PPC.

Two days later (2/27/2004) all my free listings with Overtures partners (Yahoo, MSN, etc.) disappeared. I couldn't even find my web site typing in my URL, much less my search terms.

I sent an email to the sales rep (Friday 2/27/2004 late night)at Overture and told him I knew what they had done, and they I wanted my well earned "status" back or I would contact the FCC, FTC, or whoever it took to get results.

The sales rep had told me he came on duty at 1:30 pm Mon-Fri.

All my "free site listings" showed back up the following Monday afternoon (3/1/04) at every one of Overture's partner search engines.

I received emails trying to sooth me (sucking up)and was given credit for the $149 sign up fee I paid. (He gave me a $50 discount in the beginning.)

Coincidence? NOT!

I cannot be the only small business they have done this to. Start testing them.

I signed up with Google Ad Words and now am not listed with them except for the sponsored ads. They are more stubborn than Overture was. They take my threat of reporting them too lightly.

RegisterEverywhere told me of this forum. I am starting here to expose my experience and see if others are experiencing the same exclusion.



5:36 am on Mar 7, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

>I ain't yet seen a PFI model with whatever claimed re-index refresh rate come up on it, even when url's have been changed in their programs.

Either have I. But I have seen Google fresh tags of 2/29, 3/2 and 3/5 for my pages that Google crawls for free. ;)

I vote no on PFI (in the interest of full disclosure - Y! pfi doesnt pencil out for me anyway).

I am curious though to see how the index looks at the end of April. I know which of those in my fields can afford it and which cant, and I know that those who can afford it rely on PPC at the moment and rarely show up in the free serps of either Y! or Google. I cant wait to where they show up in the serps come late April.


12:24 pm on Mar 7, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

Let's put SEO aside for a second.

Point 1. There are a huge number of websites that don't get indexed by any engine because they don't have backlinks. The owner's don't even understand that concept....they are just average Joe small business and "hobby" websites.

Point 2. For whatever reason there are a huge number of people who do not know how to use the address bar and assume that typing the domain name into the SE is an okay thing to do.....it often is.....but it is no good for those that fall foul of point 1.

For the large number of people that fall into the traps of points 1&2 above PFI is a very welcome facility. At least those typing in the domain name at an SE can find the site.

As for the vote: I am in favor of PFI for the reasons above. I am against PFI if it ever it gives an unfair advantage in SERPs. So far I haven't seen the latter.....so right now my vote is in favor!

[edited by: percentages at 1:00 pm (utc) on Mar. 7, 2004]


12:55 pm on Mar 7, 2004 (gmt 0)

10+ Year Member

Clearly, PFI does not make sense. By definition, it implies a benefit once someone pays. Therefore, there must be a negative if someone doesn't pay.

It is the ramifications of the negative on the search engine database that make PFI un-workable in the long run.

Less sites listed. Less pages within a site listed.

Searchers have already voted on this issue and it is their votes that count.


3:36 pm on Mar 7, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

Thumbs down to PFI. It can't work in the long run.

It only works as long as the search engine can drag out the indexing of new pages. That leads to a stale index. As surfers gain experience, they migrate to other engines where the content is fresher.


4:32 pm on Mar 7, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

Thumbs up for PFI. And up for Y! being the one to make it work.

Cautious thumbs up for Trusted Feeds for helping large, dynamic sites, get crawled and indexed. Yet, the model needs to change to ensure it does not bias the algo, while reading relevant and accurate content combining objective human editorial review.

Thumbs down to the CPC portion of SiteMatch, which lacks necessary controls, while being contradictory to the tenets of an algo (spend limits etc). Fundamental CPC purchaser control questions remain unanswered and unknown. ~Just imagine small biz'lack of understanding.

Clearly, Y! has made a bold move. Yet, the risk associated with this move is not as far reaching as we might think. One year ago, Y! split cpc revenues with O. And pure search was an expense. Today, Y owns all cpc revenue and PFI revenue associated with search.

In fact, Y is fully monetizing the full breath of search, shopping/commerce, local, cpc bid market, and pure search. In doing so, Y is positioning itself to capture a significant portion of this multi billion-dollar industry over the coming years. They did it because they can. They are Yahoo!, owning 28% of search market share, and being a bastion of consumer and destination value for almost a decade now. Easily enough momemtum to be the one to capture the lions share of the market. And easily sufficient precident to change again if necessary in the future.

As a professional, thumbs up for change. Up for choice. And up for the opportunity it creates.


6:43 pm on Mar 7, 2004 (gmt 0)

10+ Year Member

I guess if you're an advertiser profiting from PFI, you will be more likely to be in favor of it. If you're a user of the search engine who understands how PFI hurts the search results, you'll be against it.

Those who claim it is a good business move for Yahoo since it will make them money are thinking short term. In the end, if you lose market share due to poor results, what good was the short term gain.

How do the people who are in favor of PFI rationlize the fact that Ask Jeeves just came out and said they are dropping it because it DOES influence the search results and hurts relevancy. The quote was that it is "virtually impossible" to treat PFI the same as regular web pages.


7:04 pm on Mar 7, 2004 (gmt 0)

WebmasterWorld Senior Member heini is a WebmasterWorld Top Contributor of All Time 10+ Year Member

TrafficL, I think it's always a bit lame when professional SEOs and webmasters have to resort to appeal to the "user". You are not Joe User, I'm not, nobody here is.
If you care so much about the user, why not leave the vote to them? You have asked professional web marketers, not users.

The argument that PFI influences the serps is just as weak - what exactly is your job then? If you do it via PFI'd pages, or with good old content spamming, with buying links or producing large networks of sites - you don't mean to tell me that this is done for the user, right?

PFI is not a question of good vs evil, the question is if it works for you or not.

This 40 message thread spans 2 pages: 40

Featured Threads

Hot Threads This Week

Hot Threads This Month