homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Visit PubCon.com
Home / Forums Index / Yahoo / Yahoo Search Engine and Directory
Forum Library, Charter, Moderators: martinibuster

Yahoo Search Engine and Directory Forum

This 87 message thread spans 3 pages: < < 87 ( 1 2 [3]     
Reasons for a Yahoo Penalty
What are they? Debate the merits of each theory

 7:13 am on May 20, 2004 (gmt 0)

There's no doubt at this point that Yahoo is penalizing a lot of good sites that Google wouldn't consider to be offensive in anyway. Moreover, it would appear that you stand a substantially higher liklihood of being penalized. Yahoo is currently sending a generic email to those that inquire about a penalty that outlines the following possible reasons:

- Cloaking (showing crawlers deceptive content about a site)
- Massive domain interlinking
- Use of affiliate programs without the addition of substantial unique content
- Use of reciprocal link programs (aka “link farms”)
- Hidden text
- Excessive keyword repetition

...These reasons are generic at best. What I'm curious to determine is where the lines are and what sort of techniques could trip each of these. of these, the following seem most ambiguous and potentially far-reaching:

1. Massive Domain Interlinking - what does this mean exaclty? What is you use various subdomains for different sections of your site? Does this trigger a penalty? Too many links between each of your pages? And, how many is too many?

2. Affiliate programs without substantial unique content - What exactly is substantial? If a car site uses the descriptions of cars from its affiliate program, as part of the catalog, but the site offers their own significant car guide section - is this substantial enough? Or is there some ratio in play here?

3. Reciprical Link "programs" - this is perhaps the trickiest of them all. Google has long-since banned link farms, but could Yahoo be looking beyond simply link farms to see if a site has too many reciprical links? In a recent quick search of a popular key phrase, I found a suprisingly small number of listings with links pages.

Aside from these points, several webmasters on these threads have pointed out the following additional theories:

1. Google Ads playing a role?
2. Did specifcally PositionTech Inktomi PFI submissions raise the penalties for everyone?
3. Are certain categories being hit harder than others? Many have pointed out travel, but I've seen other examples as well.
4. Are the hardest hit categories those that Yahoo eCommerce is competing directly with?

...so come on - let's throw some examples out there, debate the theories, debunk anything we can, and get this puppy figured out! Also, if you have had any luck with getting your site reviewed and incuded, please share!



 10:54 am on May 27, 2004 (gmt 0)

I'm sufficiently arrogant enough to assume that I will appear in the top 20 for at least some of the targeted phrases! I didn't. In fact, I didn't appear unless I put in an exact obsure search term in quotes! Classic signs of a penalty.

Changed the code and - hey presto - on the first page as per normal :)

That meant that the site was spidered by slurp, not only robots.txt?
The site ever was completely banned, just lost in algo penalties?
mine ever been spidered by slurp except robots, but been spidered by yahoo vertical crawler (homepage) and scooter got an 206 for an page.

Actually I use css, and donīt think yahoo read .css, so html code donīt have any backgroundcolor nor textcolor, Could they have problem with sites completely in css?
Slurp stopped spidering my robots.txt for 3 weeks, yahoovertical crawler came by and suddenly slurp started again to go for my robots.


 11:14 am on May 27, 2004 (gmt 0)

Scooter just came back, they seems to test it, but what is this? I did put an 301 for mysite.com to www.mysite.com. Second thime gets 206 error, is that another sign of banning? - - [27/May/2004:06:41:47 -0400] "GET /robots.txt HTTP/1.1" 301 343 "-" "Scooter/3.3" - - [27/May/2004:06:41:47 -0400] "GET /robots.txt HTTP/1.1" 200 168 "-" "Scooter/3.3" - - [27/May/2004:06:41:47 -0400] "GET /espanol/ HTTP/1.1" 301 348 "-" "Scooter/3.3" - - [27/May/2004:06:41:48 -0400] "GET /robots.txt HTTP/1.1" 200 168 "http://mysite.com/espanol/" "Scooter/3.3" - - [27/May/2004:06:41:48 -0400] "GET /espanol/ HTTP/1.1" 206 2 "http://mysite.com/espanol/" "Scooter/3.3"

Seems odd, getting robots.txt from with one page as referer.

Dont know if to write again, or what all these things means


 1:09 pm on May 27, 2004 (gmt 0)

>Could they have problem with sites completely in css?

I've never had a problem using css - most sites I do use css.

Remember, my site was accepted by SiteMatch so the pages will be in the database - so I received a penalty - not a ban.


 3:32 pm on May 27, 2004 (gmt 0)


youve made some intersting comments in several threads but the one question you havent addressed and which is of real interest is the pattern of spidering before and after penalties. Was spidering normal at all times, was there a difference in spidering before and after penalties and waw the robots.txt ever the only file spidered?


 3:58 pm on May 27, 2004 (gmt 0)

As I am in PFI - I am guaranteed spidering - so I don't see the symptoms which non-PFI sites have. I always get the pages I have paid for spidered (but no others) but appear pretty much last in the results if triggering a penalty.


 5:33 pm on May 27, 2004 (gmt 0)

This is a quick note from someone who has gotten a confirmed (by Y!) penalty removed. (I say, "penalty", because the sentence "We show you have been assessed a penalty" seems clear enough...)

It took 2 months of constant haranguing with PositionTech to get confirmation of the penalty (all the while wasting good money with SiteMatch for people typing in our direct URL) to get the penalty confirmed by Y!

We got the reason for the penalty: link farm usage. Appears a marketing person put a link farm on our site for one month in summer 2003 and we had been penalized for it. We immediately sent notification to the link farm site from our corporate counsel to remove us from any and all database listings and sites less risk legal action, copying our contacts at PositionTech and Yahoo. Received an email back from the link farm site within ten minutes of sending the original stating our demands had been met. Sent the reply email to PT/Y! and our penalty was removed 2 weeks later.

We now are finally seeing our site return into the listings.

Point: It's possible to get the darn penalties removed.
Point: It takes a long time
Point: It requires a lot of harrassment of higher ups and a lot of persistence and patience
Point: It required the threat of legal action in our case.

Hope this helps someone in the same situation...


 6:59 pm on May 27, 2004 (gmt 0)

im sorry i dont follow that. You say you had a link farm on your site but had to get them to remove you? You seem to be saying it was the links INBOUND and not outbound that was the problem. This would seem to confirm what i have said before, inbound links alone can take you down.

I also hope everyone has noted....yet another expample of a site getting back in because they WERE told the exact reason for the penalty....thy keep telling us we are not allowed to know the exact reason for the penalty yet it seems many sites that have lifted a penalty did so because they WERE told the reaso/n for the penalty....when will we get a level playing field?


 7:25 pm on May 27, 2004 (gmt 0)

...had to get them to remove you

Yes, that's right. We had had no trace of anything on our site for over 9 months. In fact, the link pages were only on our site for 27 days. It was the INBOUND links from sites PARTICIPATING in the link farm that caused our removal. Is that clear enough?

...when will we get a level playing field?

Uh, if you call 3 months of constant phone calls and emails, being passed off from one person to another at 3 different companies, and finally a threat of legal action a level playing field, then, hmmm, I guess I agree with you.


 7:30 pm on May 27, 2004 (gmt 0)

To clarify on that last point, I am sure it was our having the specific link farm on our site that may have flagged us for a penalty, however, it was the inbound links that kept the penalty going. And it was the threat of legal action to the link farm company, and the forward of that correspondence to Y! that got us off the penalty list.

Hope that makes more sense.


 7:45 pm on May 27, 2004 (gmt 0)

silent..thanks..its pretty clear......

why more people havent picked up on this i have no idea....inbound links can take yo down...your competitor can take you down...thank you for confirming what i have been saying for ages.....

yes i know you had to work hard to find the reason..im just saying its seems the only way for many people..yet those who havent managed to pull off finding out the exact reason do not have a level playing field..i also ask of those that found out the reason did it help you to game Yahoo?..dont all rush at once.......


 8:16 pm on May 27, 2004 (gmt 0)

I am sure it was our having the specific link farm on our site that may have flagged us for a penalty,


however, it was the inbound links that kept the penalty going.

I doubt it. There are many here who have witnessed a 'once and out' policy.

It amazes me they wouldn't write an algo that would automatically reinstate if the offending practices are removed. Think of how many decent sites that have been penalized and are missing from the results because of the work of some unsuspecting company employee. Not the best way to present quality results to searchers.

Level playing field - it is Yahoo's stated policy not to give the reasons for a penalty, less one 'games the system'. The fact that you were given a reason, and others were not (regardless of effort), indicates there is no level playing field.



 8:36 pm on May 27, 2004 (gmt 0)

Just to emphasize: we didn't "game" Yahoo! In fact, Y! is so incredibly difficult to get a hold of (read: impersonal emails, generic responses if any at all), that we had to do most of our pushing through PositionTech.

I have never dealt with a company that has such a convoluted customer service and communication tree than Y! I simply don't understand why there are not better channels to get things taken care of. I know that if OUR company took 3 months to clear up issues that a customer was having (and, yes, we are a customer because we ARE paying for PT/Y/Overture services), we wouldn't have many customers left, now would we?

While I appreciate that Tim and Yahoo_Mike visit these forums and have set up the email address for people to provide feedback and get complaints answered, I have long had the impression that it is entirely too difficult to get a straight answer for even the most clear-cut questions (like, hey, are we penalized?).

To sum up, I guess, considering the comments of seasalt and the apparent disdain of many in this forum towards Y!'s customer service, I should just consider our company very lucky.


 9:04 pm on May 27, 2004 (gmt 0)

I should just consider our company very lucky.

not really, you pursued your goal and managed to get a succesful outcome...youre more exceptional than lucky...

and yes...i have my own evidence that inbound links ALONE can take you down....


 1:22 am on May 28, 2004 (gmt 0)

I can confirm that inbound links took my client's site down. The links date from before there were link farms. We have not had any of these files on my client's site for years now. We deleted it years back as soon as this type of file became known as link farm files. I've given up trying to get the site removed as the original file was passed around and added to other link farms. The message back from Yahoo said link files. I had already seen them listed in Yahoo as 404 files under our domain name. I could not believe they were showing as 404s since Google has never displayed them though they must have tried to follow some of the links. I occasionally saw a spider check for a page in the log files but I never found the 404 page in an index or search.

The site was in Ink until about November 2003 when it disappeared.

Yahoo has promised to review the site but my hope is fading quickly.



 1:39 am on May 28, 2004 (gmt 0)

Are inbound links from any site a problem or a specific type of site? including links that you did not have knowledge of or consent to?

OK, what qualifies as a "link farm" and, if the inbound link is without your permission, could that still cause a penalty?


 7:15 am on May 28, 2004 (gmt 0)

yes, specific types of inbounds from sites deemed inappropriate, mainly link farms or link farm style exchanges. Bottom line which i hope people will pick up on is its far easier to take down your competitor on Yahoo than Google since Yahoo does not care about collateral damage, Google does to a degree.


 1:28 pm on May 28, 2004 (gmt 0)

...will pick up on is its far easier to take down your competitor on Yahoo

Taking down competitors aside, I think it would be a fairly accurate statement that it is easier for a non-technical person to make a mistake that causes a Y! penalty than for that same person to get your site penalized on G.

So, for those members out there reading this that may be doing technical work for a company that allows marketing or creative people to make direct changes to site copy and pages, please take note.


 4:57 pm on May 28, 2004 (gmt 0)


Let's say for example that there is a web page with an 'illicit' theme and the page includes links to say, 500 web sites relating to everything from backyard gardening to steel manufacturing: would this page be considered a "link farm?"

If the example is a "link farm" would a site with an inbound link from this site be penalized?

Lastly, if the URL to the linkee is incorrect and results in a 404, is this of significance?

Daily Sparring

 5:30 pm on May 28, 2004 (gmt 0)

Res, Sounds like a farm to me. Not all farms are bad though. It is just a matter of being careful with who you allow and do not allow.


 6:12 pm on May 28, 2004 (gmt 0)

My opinion is that it is very bad. We only link to not-for-profits, professional associations and state and US agencies. We've never engaged in 'reciprocal links' and have only requested links from legitimate sites.

Furthermore, so far as we've been able to determine this web page is only found in the Yahoo database by searching on our company name/trade name.

We complained to Yahoo and received an unsatisfatory response. A further complaint resulted in the removal of a link to our site from a college, and the "illicit" theme web page has gained position.

Make of it what you will.

correction: the subject web page is also 'found' by searching on its URL.


 7:10 pm on May 28, 2004 (gmt 0)

and the page includes links to say, 500 web sites relating

er..one page linking to 500 sites?..i think the clue is right there.....


 7:50 pm on May 28, 2004 (gmt 0)

You may find this hard to believe but the number of links is somewhere in the neighborhood of 14000. The text links are numbered 1 through 14520 (there is a 14521 that has not been filled yet). The exact number is difficult to count because it looks like some links were removed. Some text links are entered multipe times - however, although the text is the same the actual linked site may be different, I've not tested. If the the page is sent to print, by the way, it's 285 pages.

I'm guessing at the number of links to legitimate sites but assume that 500 is an underestimate. You're site might be one of them!


 3:46 pm on May 29, 2004 (gmt 0)

Further observations:

Web sites with blue text on a blue background and which are included in the collection of links on "offensive site" have an Inktomi/Yahoo penalty.

Our site has blue text on a blue background, our site is included in the collection of links on "offensive site" and our site has an Inktomi/Yahoo penalty.

Searching sample of sites listed on "offensive site"
by name on Yahoo network:

A: "Offensive site" is included in search results:
-sample site has blue text & blue background
-sample site is not included in search results, indicating Inktomi/Yahoo penalty

B: "Offensive site" is *not* included in results:
-sample site is included in search results
-sample site does not have blue text & blue background

The "offensive site" displays in "A" search results in Alltheweb and AltaVista with the "family filter" or "offensive content filter" turned off.

Sample size: 10. The number of sites with penalty, and blue text and blue background: 2.

Let me know if you see a problem with above analysis or have additional sampling/testing to suggest.

Thank you!


 4:06 pm on May 29, 2004 (gmt 0)


My site is blue with blue text, not in serps.

Actually the skyblue color is the theam of the web with our logo, and text is darkblue......

But tried to take away adult filter in altavista and donīt work.

Instead of 2 sites, now there are 3 sites.


 6:52 pm on May 29, 2004 (gmt 0)

Hi helenp,

I checked the "offensive site's" link page and your site does not appear to be included.

It so far appears that a site with blue text on blue background AND included in the "offensive site" link collection is penalized AND Yahoo includes the "offensive site" link page in search results for the site.

If a site is included in the "offensive site" link collection BUT does not have blue text and blue background, the site is not penalized for merely being included in the link collection; AND Yahoo results for your site name will not include the "offensive site" link page.

Regarding the Alltheweb and AltaVista "offensive" filters; my point was that the web page that I am alleging to be "offensive" is offensive as demonstrated by the fact that the page is excluded from results for our site name when the filter is turned on, and is included in the results for our site name when the filter is turned off.

The upshot is that your site *may* have incurred a penalty for blue text on a blue background, but since you are fortunate enough not to be included in the "offensive site" link collection, the "offensive site" link page is not included in search results for your site name.

The "offensive site" link page does not appear to be in Google. Thank you Google for "Do No Evil."


 8:04 pm on May 29, 2004 (gmt 0)

I should have mentioned in earlier posts that the 'blue text on blue background' may be the cause of the "hidden text" penalty, as suggested in msg#57 above.

Regarding a site penalized for some other reason, cloaking for example, it is possible that if it is included in the "offensive site" collection of links, then the search results for the site will include the "offensive site" link page. There's no way that I can make this determination except by accident or if someone provides an example to test.


 10:05 pm on Jun 2, 2004 (gmt 0)

18 months ago all of my pages on my main site were appently given some sort of penalty by Inktomi (probably for affiliate links). Instead of being top of some fairly specific phrase - they could only be found on an exact string search and they were right at the end. About 10 days ago I finally banned Slurp from my site (they were taking a lot of pages and giving me nothing in return). Today I was investigating a jump in traffic for one particular page and I find I'm back in Yahoo (number 1 for all of the phrases). I didn't ask for any review of my penalty but I'm so grateful that I'm going to allow Slurp back onto my site!

This 87 message thread spans 3 pages: < < 87 ( 1 2 [3]
Global Options:
 top home search open messages active posts  

Home / Forums Index / Yahoo / Yahoo Search Engine and Directory
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved