Welcome to WebmasterWorld Guest from 126.96.36.199
Forum Moderators: martinibuster
We supplement this free crawl with the paid inclusion program, which allows us to add dynamic database content and other content that we could not otherwise discover and crawl. Currently less than 1% of our index is content that is included via our inclusion programs.
Our paid inclusion programs provide a superior way for sites to interact with us in a clearly structured fashion. These sites get the following
1) Control over which URLs they include in the index (subject to content
2) Frequent refresh
3) Clearly defined way to interact with us (for example: clear feed
4) Quality review and consultation (for feed customers)
5) Detailed reporting to track and optimize performance
6) Customer service
Participation in Yahoo!'s paid inclusion program does not guarantee rank in the search results. URLs are ranked in search results in the same way as all other URLs from the free crawl. The key drivers of ranking are keyword-specific relevance and site quality, as assessed by our regular search relevance algorithms with input from the quality review process.
(Note that Inktomi Index Connect clients are receiving Yahoo! Search traffic going forward, and Inktomi Search Submit customers are receiving a free trial of Yahoo! traffic until 4/15/04.)
We will also be launching free URL submission during the week of 3/1. This service allows content providers to suggest that the crawler should visit a site during the main crawl cycle. This service is different than paid inclusion in two ways:
1) There is no guarantee that we will crawl and include the URL in the index.
2) As this site potentially could be added to the main crawl, Yahoo! Slurp will follow links from that URL.
This service will reside on the Yahoo.com site and be accessible via a Yahoo! login. This service has been available for the AltaVista and Alltheweb Indices previously and the tradition will continue with Yahoo! Search.
My point exactly! And since Yahoo isn't doing PFI for their health, I bet you're gonna see the same thing there.
They are not out to help the lowly webmaster get along in this world - this is a for profit system they are building and all the sugar coating in the universe isn't going to change that...
The last 40 referrers include Google: 14, MSN: 5, Yahoo: 1.
You have probably heard this before as I believe many other site owners are experiencing the same problem as I am experiencing. Before the switch by Yahoo from Google to Yahoo and Inktomi my site was receiving a good ranking in the SERPS in Yahoo, Google and some other search engines. After the switch my site fell to the bottom of the SERPS in Yahoo for no apparent reason. It still ranks well in Google, Alta Vista and some other search engines. I had not overly optimized my site. In my opinion my site has quality content and is linked to by other quality sites. My concern and question is why am I having this problem and what can I do about it?
I have PFI in Yahoo via Net Solutions and Lycos InSite.
My point exactly! And since Yahoo isn't doing PFI for their health, I bet you're gonna see the same thing there.
This could be achieved, it seems to me, by:
--Showing *all search results* for a given query, *unbiased by PFI*,
--updating the index on a regular but not too frequent basis (e.g., every 4-8 weeks),
--updating PFI pages every 48 hours.
That way, those who want frequent spidering would need to pay. Those who don't, or can't afford it, would still be in the index, with the same shot at having a presence as the big guys.
Works for me, anyway. :-)
I can assure you that Yahoo will handle this properly and not affect free pages in the index. The only reason we will delete URLs is if they violate our content policies and are degrading the search experience.
The new program will have many benefits but higher ranking is not one of them.
I'm very encouraged by Tim's comments that in the NEW Yahoo search, if you pay to have 1 page included, they will still spider the rest of your site for free. However, since I fully understand how Ink's current PFi works, I admit that I am skeptical that this will happen since it will deincentivize people from paying. We shall see if this statement holds true.
I am also encouraged by Tim's remarks that in the NEW Yahoo search, money will NOT influence rankings since i know that with Inktomi, their Indext Connect XML feeds DO allow SEO firms to manipulate ranks in a tricky way using cloaked doorway pages. (Tim - feel free to clairfy this if its not true anymore.)
It does appear that Yahoo is listening and is starting to "come out of the closet" with how PFI really works and is trying to be more up front as to how money influences their results. Lets hope this continues.
Many of you know that I have been dealing with significant problems since I signed up for PFI.
I won't go into the deep details in this thread.
Instead, I wanted to share my overall thoughts regarding PFI in general (on a philosophical level). I don't want to touch the whole "trusted feed” issue at all (another discussion completely). I’ll just talk about search submit (or the submission of one page at a time for inclusion in a search engines database).
Even though I have had huge problems with PI, I still think PI can be an excelent model if it is done right. In theory, it can be beneficial for all parties involved;
-As a website owner, you get your pages refreshed regularly and they are "guaranteed" in the index. You also get the peace of mind (in theory) of knowing that they will always be in there.
-The searchers get up-to-date relevant data in the SERPS.
-The engine gets $.
It's a different model than Google, but it can definitely work…if it functions how it is designed to work. It will only work (IMHO) and if they mix the PI results with LOTS of freely-crawled pages. According to Tim, I think he said that less than 1% of the Yahoo database is currently PI... so perhaps they feel the same way. It remains to be seen if they are actually motivated to get out and crawl the web. Time will tell.
For me PI didn’t work how it is supposed to. Actually, it worked the exact oposite. My PI page vanished almost immediately after paying for inclusion. It’s cost me hundreds of hours trying to track down what went wrong and LOTS (and I mean LOTS) of $ in lost traffic and sales. Still, in theory, I think it's a good model and it can work if it is executed correctly and if Yahoo puts some customer service into the equation and allows you to know if your page/site has been tripped by some kind of filter or penalty. The lack of customer service/knowledge was one of the most frustrating and unfair things about Inktomi's PI system. We were hit with some kind of automated filter and our PI page (our home page which drove almost all of our traffic) vanished from Inktomi. Unfortunatley, our reselleter swore to me that there was "absolutely no penalty" at all (when in fact there was). It was the most frustrating experince I've ever had with any program/advertising I've ever had in 6 years of marketing on the web.
If Inktomi had worked how it was designed to work, I was ready to submit 40-100 PI pages to the Ink database. Ink would have made considerable $ from me and I would have been happy with my inclusion and rank. I've been holding off on further PI since my first experience was so poor, but if we can get it fixed, we will still use PI for several of our pages. I like the idea of being able to tweak your pages on a daily basis and see the results 48 hours later and also the opportunity to have fresh content in the Yahoo database every day. If something in my industry changes and there is some breaking news, I can change my PI page and get to the top of the serps for those terms as consumers search for those topics at Yahoo.
Anyway, I thought that some might be interested in hearing my honest thoughts on PI....so there you go. I guess my point is that just because I had a bad experience, doesn't mean that the model is flawed. The model can work, but Yahoo's got to figure out how to overcome some hurdles. And again, I'm just talking about "searh submit" not "trusted feed".
It's going to be interesing to see how it plays out, that's for sure!
One thing I took away from the conference was that Yahoo is intent on making this work... and they are listening to the webmaster community to get feedback on how to make it work. That, alone, is really good news. I am definately rooting for Yahoo! Even though I do VERY well, on Google, the diversification is good for the market and good for webmasters' in general.
Without the ability to tweak like the XML feeds none of the larger sites would have the least bit of interest in the product.
Can you see Amazon submitting all their ever changing pages or EBay - one page at a time through PositionTech.
Without the ability to tweak their way to the top the big players would never look at Yahoo PFI twice.
There have been a couple of other threads about your issue. I am having the same problem. Soapystar speculates that it is a duplicate content penalty caused by other sites linking to you using redirects, so that it looks like your content is on their site. This is consistent with what happened to me. Tim asked us to email email@example.com, and says they will look into it. In the mean time, see if you can find out who is duplicating your content, and see if you can get them to stop.
What guarantee? This eliminates Yahoo's ability to remove the page due to a penalty. I doubt Y! will paint themselves into this corner.
stcrim has exposed something very important:
In INK if you had a page in the db and paid for one the freebie would (or could) remain but you would never get a new freebie.
From personal experience, I believe that this is 100% correct. It is disappointing to think that because I paid for a service, my site is now flagged and is effectively being penalized (by having to pay for all future pages to be included). And it is incredibly frustrating to think that "if I had not paid for PFI, my site would be in the index". Out of principle, I will not do business with a company engaging in such deceptive practices. I hope that Yahoo does not continue this tradition.
I did see Tim's response about the new Yahoo search:
>>>Will having a few pages on PFI hurt the chance of getting the rest of that site indexed for free?<<<
No it will not in Yahoo Search.
which is encouraging, but I think that this is a topic that deserves continued attention over the coming months. I apologize that my post is somewhat of a repeat, but I wanted to support stcrim's assertion, which has profound implications.
Free search has won the battle. It's over. Yahoo PFI sounds to me like its for dynamic pages that you want for sure in the index, and for people who (basically) want to pay for spidering.
This at least is my hope: that Yahoo will get far better at free crawling and include essentially everything it should, but that you can additionally pay Yahoo for the equivalent of Google Freshbot.
Tim, you said...
"We are working on including all the quality sites in the Yahoo Directory. We have many of them but not all. Please be patient. "
Would you please elaborate on "quality sites in the Yahoo Directory." The $299 aside (which Yahoo previously kept even if site was rejected), I thought sites accepted in Yahoo Directory already went through a stringent and subjective editorial process.
Thanks for the informative feedback
1. Other sites linking to you through redirects. In this case Ink thinks that the redirect is actually the original page and thinks that yours is duplicate content spam. Then Ink penalizes your page for what another site has done (this seems especially true if the site linking to you was on the web before your site). This definitely seems to be a problem.
2. Lots of inbound links from one site in the same sphere as yours. Once again, this can be something as simple as navigation on your own site (if you use sub domains or have a Yahoo store etc.) or lots of inbound links from another site... I guess theoretically, a competotr could put up hidden or inconspicious links to your site from some random junk domain, get your PI page banned. This is theoretical... I have no idea if it would actually work.
But, these seem to be the two most reported problems. I've seen lots of evidence of both problems. If indeed these are the reasons for the penalties, it's very problematic because they allow outside influences to determine if you get banned.
I think Ink (and Yahoo is now using Ink's technology) needs to work very hard on the accuracy of their automated filtering system. Any automated "penalties" put on sites, in my opinion, should not be subject to off page factors at all and should be dependant on human intervention at all times. Otherwise, it leaves far too much room for a competitor to screw with results. Any engine that allows outside sites the ability to get another site penalized will have an epidemic on their hands within a short amount of time.
If they are going to implement automated penalties on PI pages, then they also need to provide an avenue for PI customers to have a Yahoo rep review the penalty and remove unwarranted penalties within a short amount of time (24-48 hours). Especially when they are "guaranteeing" inclusion.
I'd be willing to pay for a review service like this and I'd pay a lot of money for it too. My site is absolutely clean... I have no problem having any rep from any search engine go through it. We follow all guidelines and we are very careful to abide by rules etc. I was hit as collateral damage in the Ink crusade against spam. I can appreciate Ink/Yahoo's attempt to combat spam, but it's very frustrating when you get tagged for things that you did not do.
From what Tim alluded to in his presentations at the conference, Yahoo is well aware of these issues and they are looking at ways to solve these problems. Yahoo is progressive and forward thinking. I'm sure that they will get something in place fairly soon to solve this problem and to make the process better.... in the meantime, I'm getting killed.
I wouldn't think Yahoo would penalize a site because another site is duplicating the site's content. If that were the case one's competiter could do one great harm. I do have a few pages on my site that duplicate a few sentences, but nothing like a duplicate page.
It's not uncommon to use a tracking code on a link that runs through a cgi-bin.
No point in mincing words, Ink has completely bungled this. Yahoo needs to work hard now to fix this major problem. Quite simply they penalize the original site while ranking sites that copy content or link via a redirect. And, quite simply, you can do dramatic harm to competitors by creating such links and/or copying their content.
Interestingly, dupe content (caused by redirects or other factors) seems to be a common factor. I spent somewhere at least 200 hours (actually probably a lot more)on this issue... I've sent dozens of emails to multiple resellers and spent hours on the phone (both on hold and talking to them). I've analized serps for over 9 months straight trying to figure out what in the world happened to our site after paying for PI. I've compared sites. I called Inktomi. I've reviewed tons of sites from around the web from webmasters experiencing the same problem. I've scoured the web in search of common factors and I've found forums across the web reporting the same probelms. I've changed our home page multiple times. I've tried everything that my reseller suggested (including deleting our other highest driving page in Inktomi). I can't tell you how much sleep I've lost over this issue and how much research I have done on it.
Ink treats some redirects correctly and others it treats the dynamic link as if it were the new content. I can't figure out what causes Ink to incorrectly index some of them and to do fine with others. After all this time... it's still a mystery.
One thing is certain. We are the leader in our industry and the authority when it comes to sales and traffic. We are keyword.com (the keyword IS our industry). Our absence in Yahoo is like not having Dell show up when you search for "Dell Computers." We have never done anything to spam any engine.
We are clean.
We had two Inktomi editors go through our entire site with "a fine-toothed comb" (It's a big site) and they couldn't find anything warranting a spam penalty at all.
[edited by: kanetrain at 2:53 am (utc) on Mar. 2, 2004]
and they still didn't remove the penalty? Geeez...
First it's very hard to get a penalty from a search engine. The tricks spammers use are also very common in normal web developing. Redirects being one of the most common.
Yahoo purchased Inktomi to buy up the competition and use the technology. So what makes anyone think Yahoo PFI is going to be any different than INK PFI - after all it's being developed by the same people for exactly the same use.
All the promises in the world being offered here aren't going to make that any different. Yahoo is doing PFI for profit not to create a better database as offered earlier.
Forums like WebmasterWorld have turned out to be great marketing tools for search engines. It's a free and easy way to whip up a lot of business in a hurry.