Forum Moderators: open
In September we noticed a drastic drop in referrals from Google. Many previously indexed pages were dropped, a Google search on link: domain indicated zero, and the tool bar indicated PR=0 for internal pages. Various Google people have been quoted as saying that these symptoms are a positive indication that a site is being "punished" for "spamming". "Excessive cross-linking" is often mentioned as being considered "spamming".
A check done by an SEO professional did not disclose anything that could be considered "spamming" or any other problem with the site. However, our site is a directory and therefore unavoidably has many outgoing links just like any other directory such as Yahoo or ODP.
Emails to various Google addresses pleading for reinstatement were mainly unanswered but we did get one "form" response that claimed that the link: search "does not return a comprehensive set of results", and that we had not been penalized for spamming the index, and that we could "rest assured" that no one at Google had "hand adjusted" the results for our site. All our problems were supposedly due to changes in their ranking algorithms.
Some other larger sites such as Microsoft's Xbox site seem to be banned. Yahoo and ODP are NOT banned, link: does not equal zero, PR does not equal zero.
Now you don't have to be a rocket scientist to see that the most likely way all the various Google statements can be even "technically" simultaneously true is if Google manually "unbanned" Yahoo and ODP, letting the algorithm ban us and presumably lots of other directory type sites. Notice in particular the "careful parsing" of the emailed statement about the link: search function.
Any advice on how to correct this problem would be welcome.
It seems to me that if Google is banning directory sites (except for some), they should be honest enough to say so and indicate how they select the sites to be favored.
They seem to be trying harder to downgrade the DMOZ clones, could this be a factor with your site?
And welcome to WebmasterWorld.
1) Is it substantially composed of DMOZ, Yahoo, MapQuest, and yellow page links?
2) Review the listings of a favorite small-to-midsize town outside a major market area, and see if the guide truly reflects the town itself, or the major market, i.e., the "DigitalCities" effect. (I use "movies" as the test category --any truly local ones, or all they all 50 miles away?)
3) Do any backlinks come from authority sites (.edu, .gov, .mil, newspapers, local schools and libraries) link to them as a local resource.
NFFC brings up a good point...
They seem to be trying harder to downgrade the DMOZ clones, could this be a factor with your site?
Way too many dmoz clones out there right now.
Also, I believe excessive affiliate links are a flag of some sort. ;)
I believe excessive affiliate links are a flag of some sort.
Really page1?
Can you back that up with some facts? I would be very interested.;)
Also, I believe excessive affiliate links are a flag of some sort.
I would be interested too in hearing more, PageOne.
And, our serps actually increased dramatically since September '02.
I believe excessive affiliate links are a flag of some sort.
Can anyone back this up?
Can anyone back this up?
Unfortunately I cannot and I made the original statement. I can tell you this...
From the number of sites that I have personally reviewed over the past 8 months for a directory, I found a common denominator amongst some pages.
When it came to directories attached to web sites and those directories were penalized, there were some commonalities amongst them. One of them had to do with an excessive amount of affiliate links. Whether or not that is any factor in a penalty for directories is beyond my scope of research.
But, we all know that the major directories like Yahoo! or ODP do not like to list other directories that are mainly affiliate links. I would assume that this probably applies in other scenarios.
If I had a page with 20 links on it and 15 of them are affiliate links, I might want to consider putting those links in javascript to prevent them from being indexed. Call me paranoid, but after seeing the circumstantial evidence, I would tend to think there may be some issues.
I took a close look at the domain that I think we are discussing here and sure enough, there are quite a few affiliate links in there.
A year ago many went on a binge and started setting up link directories in hopes of artificially influencing their PageRank. Google effectively wiped out most of those directories with penalties. A search here at WebmasterWorld for link directories will probably turn up some very interesting information.
The local directories are of course processed by a program and data base system like DMOZ or Yahoo. There are about 11,000 town pages many of which have sub pages for restaurants, entertainment, etc. The intent is to try to get as much information about a particular town to users with only one or two clicks. Many small town users are on WebTV, don't spell well, or aren't very Internet savvy.
We do use some DMOZ data in our town pages along with data submitted directly by site owners. The actual URL/Title/Description data in DMOZ is excellent. The problem is the directory structure is ridiculous. You have to drill and drill and drill and when you finally get there the page is sometimes blank or has only one or two links. The alphabetic page order used by DMOZ and Yahoo doesn't have any value. Why should a site starting with "A" be ranked higher than a site starting with "Z"? Our system does ranking within a page using a search engine like algorithm and also only generates a new page if there will be a reasonable number of site listings.
We accept site listings from local site operators and post within one week. The system generates the title and description from the site's title and description tag so the quality is not as high as a human edited directory (better than a search engine) but obviously responsiveness is much better and cost to the siteowner is zero. We screen out porn, multiple submissions, all caps, etc. Users can see from the quality of the title and description what they are likely to get if they click on a link. The problem in smaller towns isn't the quality of the links. It is that there are not enough links. We get emails from folks saying "thanks for giving [townname] its first and only directory" or board, or weather, etc. We even get emails asking us to add a town so small it didn't make the cut at 11,000!
We do have some links to affiliate programs as do many larger sites. Affiliate deals are eventually going to be the best, lowest risk, way to do Internet business. We have a lot of backlinks, many from local web sites, schools, edu, etc.
Keep in mind we are talking about link: =0 and PR=0 here. This isn't just low ranking. We are completely banned from Inktomi. Google and Inktomi are indexing in excess of 3 billion and 2 billion web pages respectively. I guarantee the quality of our stuff is better than 90 percent of what they index!
Your categories...where they clearly defined or where sites in each cat very general as oposed to being specific?
I also run a directory. I do link to PR0 sites and havent noticed any adverse effects yet. I find it helps to have your categories break up as soon as possible, that way the page content remains specific to what the user may be searching for, for example dont have red widgets and blue widgets in a category called widgets, break it up into widgets with sub categories for red and blue widgets.
I can name (but I won't) 2 independent directories that use to have good PR rankings throughout. Now one has PR 2 on its home page, and all the internal pages are PR 0. The other one is showing all grey this month on all of its pages, and links from formerly PR 4 and PR 5 pages are no longer showing up in backlinks.
I am sure we'll continue to see more of this as Google continues to try an maintain the integrity of its PageRank system.
I'm interested to read other stories and/or solutions from
people with a similar problem
Another big problem is how unique the directory is especially in relation to the competition. I also loved the ODP dump a few years back and recommended using them up into last spring but agree completely that the duplicate content problems are not worth it. There are ways to make it work but I now see how important building a better breadbox is.
Is this on ODP clones or unique directories?
So I can understand them downgrading both dmoz data driven sites as well as affiliate linking sites.
RE: Javascript, I'm surprised if/that they don't use any js detection. While you can easily see what gets indexed in google in general, I wonder for their penalty flags and autopenalties, I wouldn't be surprised if they do in fact detect some basic javascript urlredirections to test for affiliate links. I would tend to doubt that if they want to penalize you, they would totally ignore a js link. But it's all just speculation I could be wrong.
I believe Google is lowering PR based upon the number of outbound links each page has. This is a simple way to reduce the importance of link farming and some directories.
I see directories which limit the number of outbound links per page keeping their PR, but those with a large number of outbound links losing PR.
In this way directories like Yahoo and Dmoz got to keep their top level PR, because the home pages have no outbound links, while the PR for deeper pages has been reduced.
I monitor a number of "top 100" sites, since September I have noticed that those with 25 outbound links per page have kept their PR, while those with 100+ links per page have dropped a point or two in PR. All other factors seem to be consistant, so I have to conclude it is purely the number of outbound links that is causing the PR drop.
Google advises keeping the number of links per page to under 100, I think if you go over that number (maybe 125) it starts to implement a PR reduction filter.
But this is why I would like more information: there is a big difference if ODP clones are getting interior pages greyed out and if good original content directories are starting to see the same thing.
If it is the later then we better start looking for a new pattern developing.
It might just be coincidence.
I have a directory site. The links all go via a cgi script (I'm not a techie - hope I put that right). There are no more than 11 outbound links on any directory page. About 700 directory pages, and each link is accompanied by review text of between 10 and 150 words. 95% of the pages share a common template, but of course content differs because of the review accompanying each listing.
I've seen my google rankings and PR rise consistently through time. While my other non-directory sites got hit bad by the Oct algo changes (PR same, but serps terrible), the directory site kept its top position for relevant keywords and continues to do well.
I've had no evidence of any penalty for being a directory.
Thanks for that information. You described it well. Sounds like everything is okay for most directories.
>>Templates
Can't be faulted for using templates, otherwise nearly all the directories on the web would get clobbered.
Must be some other factor not yet described.
They seem to be trying harder to downgrade the DMOZ clones, could this be a factor with your site?
I thought so too NFFC,
many ODP clones here [dmoz.org] are greyed out. At least when the pages go into the directory structure. However there are a few who still carry healthy PR througout the sites pages.
In any case Google seems to have weeded out ODP backlinks completely.
Google will match title and description. Rewrite both.
It is more work, but if you take the time to look over the site and write a unique description you will be serving the needs of your users too.
Hi Brad,
The neither of the 2 directories I am referring to use ODP data. Both are indepedent directories where a webmaster can add himself automatically. No editor gets involved at all. Everything is done by the webmaster adding his site, and a bot verifies the site's existence and meta-tags.
We have a lot of pages which have a mix of DMOZ data and directly submitted data. We have a few that are all DMOZ data (excluding the "clone" section which is all DMOZ data but will have META noindex tags ). We have many that are exclusively submitted data. All the pages are PR=0. The home page is PR=2.
We do not refuse site listings from free (or free with ISP service) web sites. However, a random sample of free web sites listed by us showed all were PR=4. The pages we list them on are PR=0. Free sites provide useful information especially in smaller towns. We actually offer our own free text based sites specifically designed to be easy to use for entry level users.
Paynt: We would be satisfied with the kind of treatment DMOZ and Yahoo directories are getting based on the amount of "content" vs listings, similar to ours. Link: dmoz.org shows 267,000 links. The following DMOZ page:
[dmoz.org...]
has PR=5 even though it has no content and only one listing.
This page: [dmoz.org...] which has nothing but links to other DMOZ pages, each of which in turn has only one listing and no content has a PR=6! Here we have an example of an essentially totally useless directory section which is not banned (link: 0) and has a pretty high PR. DMOZ pages with many (45+) links also had PR=6.
Neither we nor DMOZ use redirect outgoing links.
Clark: You might want to rethink doing a test like that because our whole domain is banned including pages that have only content and no external links.
GoogleGuy: Where are you when we need you?
