| This 87 message thread spans 3 pages: 87 (  2 3 ) > > || |
|Reasons for a Yahoo Penalty|
What are they? Debate the merits of each theory
| 7:13 am on May 20, 2004 (gmt 0)|
There's no doubt at this point that Yahoo is penalizing a lot of good sites that Google wouldn't consider to be offensive in anyway. Moreover, it would appear that you stand a substantially higher liklihood of being penalized. Yahoo is currently sending a generic email to those that inquire about a penalty that outlines the following possible reasons:
- Cloaking (showing crawlers deceptive content about a site)
- Massive domain interlinking
- Use of affiliate programs without the addition of substantial unique content
- Use of reciprocal link programs (aka “link farms”)
- Hidden text
- Excessive keyword repetition
...These reasons are generic at best. What I'm curious to determine is where the lines are and what sort of techniques could trip each of these. of these, the following seem most ambiguous and potentially far-reaching:
1. Massive Domain Interlinking - what does this mean exaclty? What is you use various subdomains for different sections of your site? Does this trigger a penalty? Too many links between each of your pages? And, how many is too many?
2. Affiliate programs without substantial unique content - What exactly is substantial? If a car site uses the descriptions of cars from its affiliate program, as part of the catalog, but the site offers their own significant car guide section - is this substantial enough? Or is there some ratio in play here?
3. Reciprical Link "programs" - this is perhaps the trickiest of them all. Google has long-since banned link farms, but could Yahoo be looking beyond simply link farms to see if a site has too many reciprical links? In a recent quick search of a popular key phrase, I found a suprisingly small number of listings with links pages.
Aside from these points, several webmasters on these threads have pointed out the following additional theories:
1. Google Ads playing a role?
2. Did specifcally PositionTech Inktomi PFI submissions raise the penalties for everyone?
3. Are certain categories being hit harder than others? Many have pointed out travel, but I've seen other examples as well.
4. Are the hardest hit categories those that Yahoo eCommerce is competing directly with?
...so come on - let's throw some examples out there, debate the theories, debunk anything we can, and get this puppy figured out! Also, if you have had any luck with getting your site reviewed and incuded, please share!
| 5:29 pm on May 20, 2004 (gmt 0)|
From what I've seen the reciprocal linking is penalizing lots of sites as it did ours as we aren't doing anything else to warant a penalty. If you exchange links outside of your industry or main topic you're subject to the penalty, however, Yahoo detemines what is and isn't relevant links. We actually receive orders from off-topic links as demographics play just as a significant role in advertising as topic orientation. We view links as advertsing just as Yahoo sells ads on it's homepage. Reciprocal linking is bartering and relationship building on the net and yes, it helps with rankings. You don't see Yahoo telling American Express that they can't advertise a credit card on their homepage because Yahoo isn't a "Financial Services" website.
Overall, I'm dissappointed in Yahoo because they are squashing free-enterprise and entreprenuership. Yahoo seems to panalize a website for pro-active work, but allows lots of spam for others. I know of one company in our industry who has 4 different URL's with identical sites all ranking in the top 30 and it's blatent duplication of content. As webmasters we know that Yahoo has inferior technology to Google and it's evedent in that Google doesn't need to employ a staff of humans to judge websites on their merits. Their results are as flawed as humans. This creates an un-level playing field and hurts some, but not others and when those who have been hurt do not have a remedy to have an open dialog with their punishers we all lose.
I hope that Yahoo changes their tactics as they are starting to be seen as extremely greedy and concerened on nothing but money. It's refreshing to see that Google is focused on quality and not money as with quality comes money.
Yahoo - Build it and they will come, take it and you will lose.
| 6:10 pm on May 20, 2004 (gmt 0)|
This answer from sitematch for an penalty I found in another forum:
>>>The claim they have found 2 sites that have exactly the same wording as one paragraph on our homepage so we have been banned for a duplicate content penalty.>>
I hope webmasterworld don´t get banned for this duplicated paragraph I past in.......
| 6:45 pm on May 20, 2004 (gmt 0)|
I have found that the pages that were included via Inktomi PFI a couple years ago are not included in Yahoo search. They did show for a couple of days back in March with high ranking and now are gone.
| 10:48 pm on May 20, 2004 (gmt 0)|
>>Google Ads playing a role?
Not necessarily, I've got pages with AdSense running that are doing fine.
>Did specifcally PositionTech Inktomi PFI submissions raise the penalties for everyone?
It may possibly increase the chance for human review. Didn't someone mention that 99% of the index is from the free crawl?
>>Are certain categories being hit harder than others? Many have pointed out travel, but I've seen other examples as well.
No way to know for sure if they are or aren't, but it stands to reason that quality control people might spot check some categories for human review more than others
|2. Affiliate programs without substantial unique content - |
>>What exactly is substantial? If a car site uses the descriptions of cars from its affiliate program, as part of the catalog, but the site offers their own significant car guide section - is this substantial enough? Or is there some ratio in play here?
More than likely what's in another section won't hack it if there's duplicate content on the car pages themselves. If it's descriptive content that's repeated from the "original" home site or across many sites, that could be something they more than likely would not want in the index since one page wouldn't have any more value to users than the others.
That would just be copy and paste repetition. On the other hand, if the webmaster wrote original, unique descriptions for each it couldn't constitute duplicate content because it wouldn't be. It's a page vs. page thing and if it were being detected programmatically that's how it would have to be done.
There is NO penalty currently being exacted on pages with affiliate links that have original, unique text describing the products. There are some out there doing very well. Maybe not in highly competitive industries, but there's still a difference between carbon copy or template text (and design) and original text, whatever the industry.
>>Excessive keyword repetition
Over the top keyword AND keyphrase density, and keyword/keyphrase stuffing in spots that may not be reflected with a Keyword Density Analyzer. Stuffing alt attributes, even to the extent of inserting spacer gifs excessively (hspace and vspace work equally well for images, for example. No need to throw in spacer gifs except to throw in more keyword occurrences.
Stuffing title attributes whereever possible is another way. And using excessive H(n), bolding and italicizing beyond what would be reasonable if it weren't for deliberately optimizing could be another.
Maybe not one factor by itself, but put them all together, or even just a few, and it's an obviously over-optimized site if subject to hand review, possibly coupled with other things that are just as easily detected.
If a person puts up, say, a dozen sites in the same niche, possibly with a slightly different emphases for each, and joins a "group" and gets links to all those sites from the same "group" - even if it may not technically be a link farm, it's creating an artificial linking pattern that's very easily detected.
Granted, there are sites flourishing with those types of linking arrangements, but maybe not necessarily in some search categories as much as others. It may just be that some markets are subject to stricter scrutinization - there's no way we can know except evidentially and it appears to be the case sometimes.
| 10:56 pm on May 20, 2004 (gmt 0)|
|There is NO penalty currently being exacted on pages with affiliate links that have original, unique text describing the products. |
That statement is flatly wrong.
| 11:46 pm on May 20, 2004 (gmt 0)|
>Didn't someone mention that 99% of the index is from the free crawl?
As I have previously mentioned that this propaganda is worthless. What is important is that how many of the freely crawled pages showing up in the first three pages, or the first page, or especially the top fold of first page in important commercial search pages.
I happen to check a few commercial keyphrases, and while I can see 5 sponsored listings below Yahoo's own content portal and 4 ads on the right, normally only one shows up in the organic search on my screen. Maybe my 17" monitor is too small - a 52" monitor might be able to display them all, including a few unavoidable ads at the bottom when one tries to move to page 2.
If we consider the surfers' behavior, and Yahoo knows that quite well, I would venture to guess that about 1% of the clicks in the competitive commercial categories goes to freely crawled pages. meaning 1% of the pages - all paying money to Yahoo in one form or other - get 99% of the clicks, and 99% of the pages which don't pay Yahoo, get 1% of the clicks.
This 99% free crawled page propaganda is to lure surfers to the search box by making them believe that they would be getting something relevant and then sell them its own and partners' products. Nice work!
[added]Statistics can be used to often deceive. For example, if a store during January advertises "99% of our items at 90% discount" and when we visit it we find a billion New Year's greeting cards at 90% discount but the rest - 10 million of them, selling at regular price - i.e. 10% higher than other stores.[/added]
| 1:56 am on May 21, 2004 (gmt 0)|
I couldn't personally care less what is or isn't propaganda; moreover, the general public has no idea there's even a crawl or an index. A lot of relatively savvy people out there even believe that a search engine actually goes and and searches "the web" when they do a query.
The remaining 1% that are not in by free crawl may be subject to a tighter scrutiny. The other 99% are nothing more than bits of data picked up by the crawler, whereas the PFI carry more of a responsibility since there's a business arrangement involved.
You'd have to determine whether a penalty was hand inflicted by a human or automated, for one thing. And there is a difference between applying penalties and "disallowing" things - like links that have undesirable criteria.
And for affiliate sites, there are some out there still flourishing - in there by free crawling; and in some cases comprising a full 1/3 of the first page of 15 regular results. A lot of the ones surviving do share certain characteristics, though. The ones I've personally seen doing well are what you could call "themed," and the reason for that may be that certain elements in a themed site happen to just hit the sweet spot with Yahoo/Ink optimization (still the same thing, imho).
| 2:29 am on May 21, 2004 (gmt 0)|
>I couldn't personally care less what is or isn't propaganda; moreover, the general public has no idea there's even a crawl or an index. A lot of relatively savvy people out there even believe that a search engine actually goes and and searches "the web" when they do a query.
You are right about the general public and that's why claim of 99% freely crawled pages, reiterated by Yahoo several times is dangerous.
First of all, I have discussed the matter of relevancy and I think that in commercial categories Yahoo's results are expected to be relevant because only the relevant businesses are going to pay Yahoo for favorable placement.
However, this 99% argument is for what purpose? According to FTC's guidelines, it's okay to show a list of sponsors' links only as long as this fact is stated clearly so that surfers know. While surfers don't know much about technical mumbo-jumbo, one thing right from childhood, in most developed countries, they know is that anything advertisement or sponsored has to be taken with a pinch of salt and therefore, advertisers go to great lengths to hide this fact. We all have read many product "reviews" that have the word "advertisement" almost in hidden-text form. Many "reviews" don't even have that where the link is indirect - that is if your magazine says good thing about our product we will continue placing ads is implicit.
What is Yahoo's serps results for the most competitive commercial searches? In my view it is almost-sponsored since very few clicks wll go to non-sponsored one. In entirely-sponsored results FTC will require a warning. I would say that Yahoo will keep on making the this claim of 99% freely crawled web, carefully thought out before rolling out of it search engine, not for the surfers but for the regulators who might argue that 99% sponsored is same as 100% sponsored and therefore requires a warning label, whereas Yahoo will argue that it is only 1% sponsored and the vast majority is non-sponsored, therefore needing no warning.
| 2:49 am on May 21, 2004 (gmt 0)|
>>>The claim they have found 2 sites that have exactly the same wording as one paragraph on our homepage so we have been banned for a duplicate content penalty.>>
This was the case with my site, which pre-dates the copycat by 5 years. Yahoo banned the original site, not the copycat. The copycat is now parked and Yahoo has it page 1 for my two money keywords with "Under Construction" as the heading and "This domain parked at *******.com" as the snippet.
Why do people continue to make excuses for Yahoo? Even Y! know its broken.
I dont think a debate on the 'merits' is worth the time. Most of these so-called penalties dont have any merit.
| 4:22 am on May 21, 2004 (gmt 0)|
Yahoo is an out of date technology. Prior to the introduction of their new search technology no other engine had penalized me, even the one’s Yahoo purchased. But along comes Yahoo thinking they know better than everybody else. Their answer filters that destroy the profits of many businesses.
What are they finding. Plenty of people optimize their sites, other search engines use filters to weed out duplicate content, and that sites interlink. Apparently nobody at Yahoo new that.. As time goes by they continue crucifying hundreds of businesses while they go merrily along picking up their weekly paychecks. It’s truly pathetic and nobody at that company seems to realize or care about it.
| 5:44 am on May 22, 2004 (gmt 0)|
Some of my sites that were lost are now back. Ironically the site that should have past the editorial with flying colors is still missing.
This "penalty" is making less and less sense by the day.
| 7:59 am on May 22, 2004 (gmt 0)|
It never was a "penalty" as was explained here a good hundred times.
| 8:56 am on May 22, 2004 (gmt 0)|
|It never was a "penalty" as was explained here a good hundred times. |
its a mixture of hand bans, filtered triggered bans, INK imports, filtered only and serp bugs (301', redirects etc).
There are a number of ways you wont show in the serps, some of which are lifetime bans.
| 10:11 am on May 22, 2004 (gmt 0)|
|It never was a "penalty" as was explained here a good hundred times |
You enjoy arguing semantics, and chastising other members. Yet, we rarely see any documentation to back up your claims. Can you point the interested members here to the threads wherein this was explained "a good hundred times"? We'd all be interested in seeing this, especially if the "explanations" came from a reliable source. "Penalty", "filters", they are all the same to those who no longer appear in the serps, regardless of what you choose to call it.
| 12:51 pm on May 22, 2004 (gmt 0)|
Search yourself. Both Yahoo guys mentioned the problem and said they were working on it. Semantics? What on earth does semantics have to do with it?
A penalty is deliberate. A mistake is not. Why would you even mention semantics, except to be belligerent?
Yahoo does have penalties and those are affecting some people here, but the redirect screwup was not a penalty by any rational use of the word. It was a glitch, a mistake, a failure, something broken. If you think that when you drop a glass on the floor and it breaks, that you have just deliberately penalized the glass, fine, call it that if you want. But this is a whole different type of thing than sites that are penalized.
"its a mixture of hand bans, filtered triggered bans, INK imports, filtered only and serp bugs (301', redirects etc)."
Sorry, no, and that is the point. The penalties affecting people are completely different than the woes caused by the mistake. Yahoo is working to fix the mistake. They will continue to have penalties.
| 1:06 pm on May 22, 2004 (gmt 0)|
|"its a mixture of hand bans, filtered triggered bans, INK imports, filtered only and serp bugs (301', redirects etc)." |
Sorry, no, and that is the point. The penalties affecting people are completely different than the woes caused by the mistake. Yahoo is working to fix the mistake. They will continue to have penalties
the reasons for sites not appearing are indeed a mixture of the above mentioned. You then went on to mention both penalties and mistakes so its not clear why you say no and then agree with me. You said it was mentioned a hundreds of times that people not appearing was because of bugs or what you call redirect problems (actually theres more than just redirect problems thats why i term them all under bugs). The point of course is that SOME peple arent appearing for that reason, others for other reasons including real penalties, many people arent aware of which problem they have, they just no they dont appear. You seem totaly focused on one single bug, the redirect 301 problem, there are hundreds of thread mentioning other problems and reasons for appearing in serps.
| 1:34 pm on May 22, 2004 (gmt 0)|
soapystar, the problem here is confusing two completely different things, both of which have the same result.
I'm not completely focused on anything. I am interested in newbies not being confused by the handwaving in some of these threads. The redirect/duplicate bug effects a lot of people, and despite the Yahoo guys saying several times they were working on the problem, the armwaving goes on.
Look at the first post, and then #12. It only doesn't make sense if you think it was a deliberate penalty. There are no "reasons" other than Yahoo had a technical failure. THAT is the reason. The only thing to do was wait (unless you had reason to think you were causing your own redirect or duplicate issue).
And this has *nothing* to do with the penalties Yahoo is assessing on sites. Different issues, different topics, different threads.
It is true that someone whose site(s) are not appearing may not know if this is because it/they falls into the "penalty group" or the "bug group", but they are NOT the same group. crobb305 saying it doesn't matter why you aren't appearing, that everything is just one big "let's whine at Yahoo" group, is just not a very helpful way to look at things.
| 2:14 pm on May 22, 2004 (gmt 0)|
some of these sites are coming back after an algo change. This suggests to me that a filter hs been relaxed rather than anything fixed.
| 2:49 pm on May 22, 2004 (gmt 0)|
Steve, there *is* a limit to how much whining can be done without people going daffy. With some sites there is *no* way to tell what the problem is without assessing the individual site - and that just ain't gonna happen, no matter how much any particular individuals want to whine to make it happen.
| 3:05 pm on May 22, 2004 (gmt 0)|
Man, I thought you guys would be excited by the news that several of my sites came back - surely others did as well then - I'm talking about 5 unrelated sites. Anyone else see their sites back where they were pre-March15 or pre-May1 (the 2 dates that seem to apply)?
| 4:08 pm on May 22, 2004 (gmt 0)|
Penalty vs Filters etc.
For me, it has been important to know which one was the cause of my site's absence from Y. I know that my site has duplicate content which has been there for several months and will have to stay there for a few more weeks. Google has kept the site high in the rankings for both pages, but the whole site vanished from Yahoo when Y re-launched.
I have been monitoring the threads on Y's penalty/filter situation with great interest.
Cabbagehead - My site came back on Y last week. One of the two duplicates shows up nicely on page one of Y's SERP. The other page is nowhere to be found.
Filter, not penalty in my case.
| 4:08 pm on May 22, 2004 (gmt 0)|
I am very pleased for you,
my is not in and none of others I know of.
I think they just changed their algo, and cleaned up old cached pages,..at least my backlinks pages,
some of these old cached pages from 2003 could be reason for my penalty or not getting included, just hope we all get in soon.
The sites that now are back in, was they spidered by slurp regularly?
Now 3 weeks has gone since slurp visited my robots.txt
| 6:52 pm on May 22, 2004 (gmt 0)|
I am glad your sites came back cabbagehead, my site was dropped on May 1st too, except they still let 2 or 3 pages show up for terms, that is all. What is the May 1st fiasco anyway? What changed?
| 10:43 pm on May 22, 2004 (gmt 0)|
"Man, I thought you guys would be excited by the news that several of my sites came back"
Unfortunately it seems some folks with penalties are not happy that the technical problem is being addressed because "politically" it was more whine-productive to lump them together. For me five of six things were fixed. Maybe cabbagehead's percentage is about the same. This has nothing at all to do with a algo change.
cabbagehead, perhaps you will send feedback to yahoo pointing out something that was not fixed, in comparison to something that was. They might still not have 100% of the problem addressed.
| 3:02 am on May 23, 2004 (gmt 0)|
I had one site return. I had two other sites that returned for 24 hours a week or so ago, then disappear again. These two sites had old Inktomi "penalties" (or whatever steveb would call it). I am not sure why the sites reappeared for 24 hours, as if the "penalties" were removed, then disappear again. Maybe some changes in the works that will come out over the next few weeks.
Oh, also, in your original list of POSSIBLE reasons for "penalties" you didn't mention the redirect urls that some members have discussed in other threads. Some suggest that there could be a "filter" or "penalty" for multiple urls pointing to the same website. One of my domains seems to have suffered because of this since there are 20+ redirect urls pointing to it outside of my control.
| 3:23 am on May 23, 2004 (gmt 0)|
Soapy, you hit the nail on the head when you said
|its a mixture of hand bans, filtered triggered bans, INK imports, filtered only and serp bugs (301', redirects etc). |
For Steveb to say so definitively that "there IS no penalty" is a sweeping generalization. In some cases, there are penalties. There are algorithmic penalties stemming from the 301 redirect problem, and old human-imposed Inktomi penalties. I am not sure what, exactly, steve is referring to when he says "NO PENALTY".
| 4:17 am on May 23, 2004 (gmt 0)|
"In some cases, there are penalties."
I said that many times, so why be obtuse about it?
I don't know why you insist combining apples and oranges and calling them all oranges. It makes no sense.
Some sites are oranges, and have been deliberately penalized. That is one discussion.
Some pages are apples and are mistakenly lost.
That's it. It's not complicated and there is no sensible reason to confuse the issues.
More to the point, people who have been penalized could conceivably get Yahoo to help them. People victim of the technical mistake (up to recently anyway) could NOT get help from Yahoo under any circumstances. Yahoo was TRYING to help, and still is. Complaining about being unjustly penalized is one thing. Complaining about being broken when Yahoo is trying to fix it is more than pointless.
My comment to cabbagehead (about his specific comment) that it never was a penalty is exactly right and you should just admit that and move on. The thing he was talking about -- which is different than things others are talking about -- was not a deliberate penalty.
| 7:02 am on May 23, 2004 (gmt 0)|
Well, tonight I think I've joined those here in misery.
My site is gone from the Yahoo serps for all of my normal search terms.
I think I've received a duplicate penalty because of my Affiliates. I have a Yahoo Store, and each affiliate gets a link like this:
Since Yahoo dropped google and started it's own search, these affiliate links have been showing up in the Yahoo serps. Sometimes even higher than my own page. They point to my homepage and would appear to be mirror sites. I've been worried for a while now that the new Yahoo algo was going to see them as duplicate content.
It's amazing that Yahoo SEARCH has done this with their own Yahoo STORES. Yahoo Store's affiliate program creates these URL's automatically to send to our affiliates. Yahoo encourages the store owners to get affiliates, but when we do, they penalize us for duplicate content.
Of course, this is just my guess on why I was dropped from Yahoo, and maybe I'll get lucky and be back in tomorrow. But regardless, Yahoo shouldn't have all those duplicates of my site in their index. Google seems to sort them out just fine. Why can't Yahoo?
I was having the same problem with Inktomi, but even they seem to be sorting them out better than Yahoo.
Guess I'll be searching for a new store host.
| 7:07 am on May 23, 2004 (gmt 0)|
Searched again, and I'm back in Yahoo..
However, I see I'm gone from Altavista and Alltheweb.
Same thing happened earlier. I think it's just a different datacenter showing earlier data. I really have a bad feeling that I'm gone.
| This 87 message thread spans 3 pages: 87 (  2 3 ) > > |