Forum Moderators: open
We supplement this free crawl with the paid inclusion program, which allows us to add dynamic database content and other content that we could not otherwise discover and crawl. Currently less than 1% of our index is content that is included via our inclusion programs.
Our paid inclusion programs provide a superior way for sites to interact with us in a clearly structured fashion. These sites get the following
benefits:
1) Control over which URLs they include in the index (subject to content
guidelines)
2) Frequent refresh
3) Clearly defined way to interact with us (for example: clear feed
specifications)
4) Quality review and consultation (for feed customers)
5) Detailed reporting to track and optimize performance
6) Customer service
Participation in Yahoo!'s paid inclusion program does not guarantee rank in the search results. URLs are ranked in search results in the same way as all other URLs from the free crawl. The key drivers of ranking are keyword-specific relevance and site quality, as assessed by our regular search relevance algorithms with input from the quality review process.
(Note that Inktomi Index Connect clients are receiving Yahoo! Search traffic going forward, and Inktomi Search Submit customers are receiving a free trial of Yahoo! traffic until 4/15/04.)
We will also be launching free URL submission during the week of 3/1. This service allows content providers to suggest that the crawler should visit a site during the main crawl cycle. This service is different than paid inclusion in two ways:
1) There is no guarantee that we will crawl and include the URL in the index.
2) As this site potentially could be added to the main crawl, Yahoo! Slurp will follow links from that URL.
This service will reside on the Yahoo.com site and be accessible via a Yahoo! login. This service has been available for the AltaVista and Alltheweb Indices previously and the tradition will continue with Yahoo! Search.
All search engines crawl the web for free. Crawling and indexing are two completely different things.
>>>Their intention is to find every public page available and put it in the index.<<<
Does public page equal .edu and .gov or does that mean that will include amazon.com or edmunds.com in their free index? Could someone (from Yahoo)define "public page"?
Sounds like if you want to buy an apple pie, Google is the way to go - but if you want the safety rating on a child seat --- well I'd still use google because I like being the one to decide what's valuable to me and what's not.
-s-
The big question is...
Will the free inluded URL be treated the same as the paying URL?
If not then your free URL inclusion is just actually helping Yahoo to make money out of it in exchange for mediocre traffic coming from them.
Okay, let's talk economics ;)
You have 10,000 pages and Yahoo crawl all of them for free. Not only that, it would even come back to refresh as often as you refresh your pages.
In return Yahoo gives back 1% or even less in free traffic each month.
If you are happy with that kind of arrangement then good for you.
But that won't work with me. My minimum exchange ratio with any SE is at 10% other than that I'll throttle its access. I'm not paying for bandwidth just to provide Yahoo free content and fillers.
Nor would I pay for 15c or 30c for questionable traffic.
Will the free inluded URL be treated the same as the paying URL?
I would say yes, going by how Alta Vista ranks paying and none paying sites. plus with Ask Jeeves going the same route I think Yahoo must stay competitive.
It appears that signing up for PI with Ink and now Yahoo puts your site in danger of getting hit with the "vanishing PI homepage" ban. There are a few different names for the same problem. But several webmasters have reported that signing up for PI with you will put our sites in danger of some type of automated filter or penalty that can't be removed and never will.
Apparently even the resellers of PFI don't even know about it or how to fix it. Or they know about it and deny it because you guys don't know how to fix the bug.
How in the world could we trust a program that seems to have penalized so many legitimate sites without any explanation or recourse?
I'm personally scared to death of PI. Once you sign up, if you get hit by their filter you are gone for good and nobody seems to know how to fix it. I will not even consider Yahoo PI/PPC until this bug is fixed.
I know that my site doesn't have any spam or hidden text or anything, but it seems that other unspamming, clean authority sites have been hit by this penalty for no apparent reason or explanation.
1) keywords not competitve. No (or little) Overture, no need to pay PFI, simple optimization should bring you to the top.2) keywords moderately competitve. Overture listings will take almost all of the searches. Few people will scroll down to regular SERPs. No need to pay PFI.
3) keywords highly competitve. Overture listings take most of the searches, but still in absolute numbers a lot of people venture to the regular SERPs. If the words are so competitive, the margins should be high and PFI should be a bargain.
My understanding is that Yahoo will only show 3 - 4 Overture listings before normal web results. Tim has stated that SiteMatch listings will be mixed with free listings and not given a relevancy boost. If that's the case I think your scenarios 2 & 3 won't apply.
Scenarios 2 & 3 are more like the situation you could get in MSN. i.e. when if there were a lot of LookSmart results Inktomi listings were pushed back so no matter how well a site was optimised and ranking in Inktomi you still didn't make the top 30 results.
Under this new system, that would make a $.30 click have an effective cost of ~$.35.
seasalt
...there is an online demo of the Site Match interface available...
For some reason, any account management center that uses the phrase "Fund Depletion" just sends a shiver up my spine.
seasalt
For some reason, any account management center that uses the phrase "Fund Depletion" just sends a shiver up my spine.
lol - allthough they seem to have made topping up again quite straightforward. ;)
seem to have made topping up again quite straightforward. ;)
Yes, they seem to have made *that* service free. ;)
Fact, Yahoo have increase the amount of spidering on all our sites
Fact, Yahoo are a business just like mine and yours.
Fact, when Yahoo switched to Yahink we where in the main all happy.
Fact, Yahoo are pushing more traffic to us then ever before.
Lets give the guys at Yahoo a chance to prove what they are doing will compete with google, instead of just bashing them.
DaveN
OK, now I feel badly about my useless quip in #345.
That said, there are a lot of legitimate questions that do need to be asked, and answered, as webmasters finalize short term planning for the next several months.
Fortunately, Tim is stepping up and handling those, in finest tradition of WW. :-)
DaveN
I think the issue here is not Tim, though he's being somewhat targeted - it's the Y program. But he's use to it - he was sitting in the same seat as an employee of Inktomi a few years back when they took this slippery slope.
For most trying to make a living on the net, this just isn't affordable.
My sites are only worth a few cents per visitor, but they get thousands and thousands of them. So I for one will avoid Yahoo.
For those few that can make a living after Yahoo's cut, I say have at it...
-s-
all I want see is a fair deal for all, and i think that Y! do too, at least thats the feeling i got from tim.
DaveN
I'd like to hear a guarantee on that from Tim or Yahoo.
If payment doesn't affect ranking, there is no issue. If it does, at all, there is a serious issue. Not only for all you guys who are trying to make living, but for Yahoo itself, which may very qickly find itself in the same position MSN was in a while back with a diminishing market. Who wants to search on an engine which is dominated by payment rather than pure relevance? Quality non-commercial sites (like ours) would be buried.
Also, wasn't there something from the government about separating pure returns from those influenced by money?
I'd like to see a clear definitive statement on this from Yahoo. If there is nothing to hide, why not clear the air and say so?
Participation in Yahoo!'s paid inclusion program does not guarantee rank in the search results. URLs are ranked in search results in the same way as all other URLs from the free crawl.
I was involved with the creation of the first PFI programs, so i know the technical aspects that others may not be aware of and would like to quickly explain the most important aspects.
There are 2 types of PFI:
1. single URL submission
2. XML data feed submission
If done properly and honestly, the single URL submission would not affect the relevancy of the search results since thespider is looking at the actual page that exists and isa pplying the normal algorithm on it and letting that page rank where it may. (I am still skeptical, however that engines do not give these a boost).
The XML data feeds are entirely different. Rather than submitting real, actual web pages, SEM resellers simply take keywords and auto generate massive amounts of "false" pages around those words. These pages do really exist online. The SEM company has the ability to create the Title tag, the description, and the body content.
These pages are fed directly into Yahoo's index. When someone clicks on one of these links, they DO NOT EVER SEE THE FAKE PAGE THAT WAS SUBMITTED. They are redirected to a designated URL within the data feed on the client's actual website - such that no one ever realizes what is really taking place.
While the algorithm is applied on these XML feeds, they are not handled the same as the rest of the web. Here is what i mean:
1. Since the pages are auto generated, they do not have any Link Popularity. This fact could cause the pages to rank very low, so rather than letting that happen, Inktomi applies the same Link Popularity from the REAL site to these fake pages... giving the pages a boost.
2. Since these company feeds can so easily be changed all at once by the SEM reseller, they can very easily tweak their pages in mass until their pages are ranking higher and higher. Since the feed is refreshed so often, they just keep tweaking all the "false" data in the feeds until they hit.
3. As the engines are getting more intelligent and looking at communities and semantics to determine relevancy, it has become more and more impossible to gauge the TRUE relevancy of an auto generated page since it DOES NOT REALLY EXIST IN ANY COMMUNITY.
The recent decision by Ask Jeeves to drop its XML feed PFI program, but keep their Single URL program backs all these points up. In fact, Ask Jeeves explains why the XML feed pages were "VIRTUALLY IMPOSSIBLE" to use but not treat differently than "REAL" pages.
Read what they say here in this press release:
[clickz.com...]
"Concern about how paid inclusion data affected its search engine's relevance led Ask Jeeves to drop its Index Express XML feed program. The company also says the initiative wasn't paying off as expected.
"When I say it impacted relevance," said Jim Lanzone, vice president of product management at Ask Jeeves, "I mean both good and bad. Sometimes [something ranked] better than it should have ranked, sometimes worse."
The company introduced its paid inclusion services in June 2002, but says it's since come to the conclusion it's virtually impossible to merge structured data, such as that in XML feeds, with the unstructured data that comes from crawling the Web.
"It is difficult for the algorithm to understand types of information it's not used to. When we put this information in, the results sometimes showed up in the wrong place," said Paul Gardi, SVP of operations and strategic planning at Ask Jeeves."
I summary, i believe that Yahoo will soon be forced to follow the lead of Ask Jeeves and do away with their XML PFI program. The only reason they have gotten away with it for this long is because people do not understand how the technology is really working and causing the results to be affected.
-s-
DaveN
Tim posted this morning in another thread the new URL for the YAHOO FREE SUBMISSION.
You do have to log into Yahoo first. I have tried it and it seems to be working.
Taking a look at the AskJeeves chart (NASDAQ:ASKJ) and its monumental move this morning, you can see with clear direction that this sector is consolidating.
I would have though this paid inclusion thing and pay per click revenue model would have caused a stir but not at this magnitude, it is like 1997 all over again.
YHOO makes it's PFI move while ASKJ takes theirs away. Google sits tight thinking the YHOO move in PFI is a bad test program.
Now you have the Yahoo inclusion program readying the use of Viewpoints (NASDAQ:VWPT) tool-bar for better results and a better user exp. All these companies have huge potential; this is shown in the way they are bahaving for their investors.
Is it smart what YHOO is doing? I really do feel that it is not but this is just my opinion, they really need to put out some press to clear this all up, I think they want to see all of our reactions first thus the delay of clarification. It's like serious free feedback from the webmasters community.
You have to wonder which guy will produce a failure in this ground-breaking search environment that mimics the fall of the Netscape browser.
Hollywood.
SO IS regular Yahoo.com but it depends on how you enter in your search term. For example:
Instead of putting a space between your words as a search on Yahoo, add in a "+" operator between your words, and the SERP will be identical to a search done on Google for the same words minus the operators.
Also, I know without question that it's very very current.
It depends who's side you are on, Is Yahoo right or is Askjeeves wrong, my vote is with Askjeeves and not Yahoo as I think a good user exp is the best way to tweak a search engine.
It is very clear to me (In my opinion) that Yahoo is using the broad scale approach and trying to earn more revenue by trying to persuade users to get use to pay for inclusion/pay per click combined while Askjeeves thinks a happy surfer is from a better engine algorhtym. I trust Akjeeves more.
Hollywood
=================== in re below ====================
[clickz.com...]
Ask Jeeves brought in $2.1 million from paid inclusion in 2003, a small sum compared to the $77 million it generated from paid placement. The company carries Google AdWords listings on its sites. "Even if we had all those queries, I don't think it's worth doing," said Lanzone. "I think there are much better ways.... There are probably better mousetraps in dealing with structured data."
Lanzone also noted that, given the other drawbacks, it was easier to drop the program because of the controversy that surrounds XML paid inclusion.
"I could see why people would be suspicious of it," he said. "They would assume that where there's smoke there's fire. I don't want our integrity called into question."
Ask Jeeves will continue to offer the Site Submit program, under which smaller advertisers pay a flat fee to submit their URLs to be crawled.
1. Relevance. This is the main reason why we are eliminating Index Express. After considerable testing, we found that this program impacted the relevance of our search results. The reason, we believe, is that Index Express introduces structured (inorganic) data into an otherwise unstructured (organic) Web environment and the inconsistent data feeds are difficult to rank. Since our top priority remains providing users with the most relevant search results on the Web, and we have worked hard to achieve world-class search results over the past two years, we made the decision to eliminate it.
2. Revenue. We determined that Paid Inclusion – as it exists today – is not the most efficient monetization vehicle for a search engine and dropping Index Express will not have a material impact on revenue for the company. Additionally, we know that user experience drives searches, and increasing search volume is ultimately the best monetization strategy.
3. There must be a better way. Ask Jeeves supports some of the fundamental premises that paid inclusion is based on, notably the need to acquire deep-web content, and the right of search engines to share in the value they create for businesses. However, we have concluded that our Index Express program is not the right model to solve either of these issues.
It is also important to note that while Ask Jeeves is eliminating our Index Express program, we are keeping our Site Submit program in place. Site Submit allows web site owners to pay a fee, per URL submitted, for immediate inclusion of their site into the Ask Jeeves search engine, if the site passes inspection (sites are rejected for spam, porn, etc). Some customers also choose to have their sites refreshed on a more frequent basis. These are actual, organic Web sites who are simply paying to shorten the amount of time it takes for our engine to crawl their sites. We believe this program will continue to be useful to both search engines, users, and site owners. We also believe the model for this program avoids the pitfalls of Index Express, which is why this program has not historically been a hot button issue for those inside or outside the industry.
