Forum Moderators: open
We supplement this free crawl with the paid inclusion program, which allows us to add dynamic database content and other content that we could not otherwise discover and crawl. Currently less than 1% of our index is content that is included via our inclusion programs.
Our paid inclusion programs provide a superior way for sites to interact with us in a clearly structured fashion. These sites get the following
benefits:
1) Control over which URLs they include in the index (subject to content
guidelines)
2) Frequent refresh
3) Clearly defined way to interact with us (for example: clear feed
specifications)
4) Quality review and consultation (for feed customers)
5) Detailed reporting to track and optimize performance
6) Customer service
Participation in Yahoo!'s paid inclusion program does not guarantee rank in the search results. URLs are ranked in search results in the same way as all other URLs from the free crawl. The key drivers of ranking are keyword-specific relevance and site quality, as assessed by our regular search relevance algorithms with input from the quality review process.
(Note that Inktomi Index Connect clients are receiving Yahoo! Search traffic going forward, and Inktomi Search Submit customers are receiving a free trial of Yahoo! traffic until 4/15/04.)
We will also be launching free URL submission during the week of 3/1. This service allows content providers to suggest that the crawler should visit a site during the main crawl cycle. This service is different than paid inclusion in two ways:
1) There is no guarantee that we will crawl and include the URL in the index.
2) As this site potentially could be added to the main crawl, Yahoo! Slurp will follow links from that URL.
This service will reside on the Yahoo.com site and be accessible via a Yahoo! login. This service has been available for the AltaVista and Alltheweb Indices previously and the tradition will continue with Yahoo! Search.
Based on a conversation with Tim, he said that these pages would continue to be in the index, subject to spam guidelines, etc.
So for those of you who have waited forever for inclusion, if you are in Yahoo now, you should remain so in the future.
As far as PFI goes, Tim specifically said that PFI people are getting what they paid for -- inclusion in the then-current INK network. The Yahoo inclusion is a bonus.
Right or not, nowhere did I see in PositionTech that Yahoo was included.
Any idea what will happen to UK customers?
Have we been forgoten?
"60 days of free traffic" that finishes on 14/4 would mean INK results should be included by now but uk.search.yahoo.com is still showing google results.
Help appreciated.
When you say the Inktomi Network, MSN is basicly the only thing of value and they are spidering and getting ready to take on Google and Yahoo for the sport of it.
Why would anyone want to pay for that knowing it's going to end soon?
And, isn't yahoo going to face the same "very poor" search results Inktomi offered by delivering paid results?
And, if you can get in for free if you wait - don't you think most people will wait?
In all fairness to the searcher, will they be made aware the results that are viewing are paid?
-s-
Worse still our PFI urls went to the back of the results. A simple check would have revealed we are a top 3 market leader, the competitor is not and has duplicate sites and the spoof domains used resolve to blocks of free ISP DNS they used.
Do Yahoo now want us to pay them to correct this?
In all fairness to the searcher, will they be made aware the results that are viewing are paid?
Who said the new Yahoo PFI will get a site ranking higher? Why would the searcher need to be notified? Tim spelt out clearly in message 1 what the paid service involves...nothing there about being ranked higher or listed seperatly.
From what I understand, if there are good links to the site, you will be in the index (Yahoo's webamster guidelines say that) and I assume that most of us will not need the Yahoo paid service. (same as the Ink paid service - nothing to do with rankings, just a gaurantee you get crawled every 48hrs - if you alrady rank well at Ink, why bother paying?)
There is a great deal of concern for those in the Google AdSense program that Yahoo will treat those pages as affiliate pages and drop them from their free and paid database...
Can you address that situation...
-s-
At the end of the day, the fact remains that Yahoo has decided to continue sacrificing the quality of their index in order to make a quick buck.
BOTTOM LINE: PFI creates an inherent incentive to NOT keep the index as fresh and complete as possible since doing so would mean no one would ever pay to be included!
The only way they can create an incentive for people to pay is by claiming that their own engine is not good enough to index the whole web on its own (like Google does), so the only way to ensure that you'll be in is if you pay.
It's great to be hearing Tim and Yahoo reps starting to talk freely about increasing their efforts to spider the ENTIRE web. Up to now, they never talked much about aggresive "free crawling" because they new it would make people feel less of a need to pay. They're spin appears to be that their new PFI program will only consist of content that they "could not find on their own".
Why can't they find this content? If they spent all the energy they're putting into this PFI program towards making their spider and index better, they would be able to find all this so called "hidden" content and refresh their index faster... which will create a much better user experience in the long run.
Dont get me wrong. I am actually a big fan of Yahoo. I'm very impressed with their new engine and am glad they've created some genuine competition in the space. It will only help the industry and the users in the long run. I'm fully confident that in the long run, as users and the general public become more aware, Yahoo will dump their PFI program and will choose to focus rather on building the best possible search engine.
Very intresting point of view TrafficL. That does make us all think about how search engines can be more lucrative. However, there is still a need for many sites that have URLs longer than the Chinese Wall and . . . YES, there are still many webdesigners out there that love to do pure FLASH sites and don't understand SEO (and perhaps don't care) because ART is more important than what "view source code" is and how it is viewed by machines (spiders). Therefore, a PI program is a must need for many website owners/designers/etc and a great concept to offer by Yahoo! Perhaps it's things like this that will let them win back the lead.
On a different note:
"2. Customers who have already paid for the current Inktomi program will continue to get all the benefits they have paid for - distribution across the pre-Yahoo! Inktomi network. As an additional bonus, they will get 60 days of free traffic on the new Yahoo! distribution."
I just hope reselling partners (such as PositionTech) will extend the "60 days of free traffic" to its clients . . . or is it Pay Day for them?
Your support, feedback and answers to specific questions should be commended. Thanks to Y also for allowing your indulgence in WW
Regards
EW
If you are currently in the index and all looks well I would continue creating more great content that we can crawl for free. I think the only change you will see in the future is greater comprehensiveness and freshness.
These kind of comments fill me with confidence, especially,
is greater comprehensiveness and freshness
I do feel Yahoo can catch up and genuinely challenge Goolge. And it may not take too long. What they need to do is make sure the cream rises to the top a bit more.
I have been quite a critic of Kelkoo's top ranking in Google, but I have to say in the commercial serps I think Yahoo could do with sites like this ranking better. Because what is beating Kelkoo is far worse in terms of quality and relevancy.
Than who in their right mind would pay for it?
-s-
Can you give us an example or two of content that your search engine spider currently is not able to find on its own?
... and assuming there ARE examples of that type of content, why not just improve your spider such that next time it CAN find that content on its own?
In my opinion, relying on PFI to find this content is not the best way to ensure that this "hidden" content finds its way into your index. The only true way is to build a spider that can find it on its own. This is the route Google seems to be taking.
Also, if your spider does improve such that it can find this content on its own, would you discontinue this PFI program, or would you purposefully not let your spider include it since you would than be turning down payment for its inclusion?
(Tim - i appreciate that Yahoo has given us the opportunity to interact with you and provide feedback like this. You've been a great source of feedback for us all here at WebmasterWorld. I am NOT critical of Yahoo... i think they're great, I am just a firm believer that PFI will hurt Yahoo's reputation in the long run. I understand that other side of this argument and I think this is a healthy debate.)
The key drivers of ranking are keyword-specific relevance and site quality, as assessed by our regular search relevance algorithms with input from the quality review process.
quality review process?
To me, it looks like a big bloated yahoo shopping index than a search index.
PFI pages don't get a boost
This is true. I have a redirected page pointing to a PFI page and because of the way Ink works, both URLs are listed seperately in the database, though it's the same page. The PFI page is 13 places lower than the redirected page.
On a more serious note, how come you are not Yahooman? ;)
My site has over 50,000 "eyeball" page views a day (all bots are excluded from this count), and has 120,000 static pages. Google has covered it fairly well now for three years, by including anywhere from 60,000 to 90,000 pages from this nonprofit, tax-exempt site in their index.
For the last year, Slurp has had only 400 pages in their index from this site. What's curious is that the site has no competition, and there is no other way for anyone to find this information on the Web. But Slurp doesn't care. The question in my mind is how committed will Slurp be when it comes to deep-crawling the noncommercial sector? I'm not optimistic.
Right now, I don't need Yahoo. It seems that most noncommercial information seekers are already conditioned to use Google. When Yahoo dropped Google, my page views dropped only about eight percent.
As for a promised review of this matter by Yahoo. PT suggested I contact about 15 search engines and request removal of my listings because Inktomi viewed this as duplicate content. Am I going to be listening to the same craziness from Yahoo hourly employees, with a review process. Plus suggestions to alter my site that has never been penalized with any other engine.
Quite frankly I'm seeing bullentin board sites in Yahoo with 10,000 pages of spam and I can't get a small site in.
Care to comment Tim on what happened here? Was that reltaed to the new search being rolled on on Yahoo as well as MSN
Doug
Customers who have already paid for the current Inktomi program will continue to get all the benefits they have paid for - distribution across the pre-Yahoo! Inktomi network. As an additional bonus, they will get 60 days of free traffic on the new Yahoo! distribution.
Actually, I paid (on Tuesday, no less) to be in Yahoo. Seeing a drop in traffic I clicked on the "what's this" link on a Yahoo SERP and read the following text:
Web Results are the most relevant web pages found by Yahoo! Search Technology in response to your search terms. Web Results may include sites that participate in the Inktomi Index Connect™ or Search Submit™ programs. Sites from both of these programs pay to be included in our search engine index, and for Index Connect, sites also pay to be reviewed for relevance.
So I followed the links and signed up for a year. But that's now what I'm getting.
I think I will call my credit card company tomorrow and dispute the charge.
ATAadams- If you could be more specific I could maybe answer your question. Is your post a question or a commentary?
Doug
