Forum Moderators: Robert Charlton & goodroi
If you run a web directory, feel free to post your experience here.
I suspect you've been caught by some dupe filters.
Or .. [dmoz.org...]
As far as I can tell its been PR0ed for a while.
The site is about 2 years old and is squeakey clean as far as SEO goes, but today it is gone from Googles SERPs and is now PR0.
I'm not concerned for the loss in traffic, because it pretty much gets its own traffic from regular users, but I would like to know if I did something wrong. I sure hope that it doesn't happen to any of my other directory sites, but without knowing why one site was de-indexed, I don't think it would be wise to make any changes.
Are there any other directory site owners who have had their directories de-indexed lately?
Could my site be confused as a scraper site?
zoltan, could you site be in a similar situation?
Possible if some knucklehead reported it as such, and someone at Google gave it a cursory look and flipped the switch on it.
I have a hand edited business directory that takes free submissions, it charges no fees, that I filled up using my own knowledge of local businesses. Nothing black hat about it. It went gray bar in April.
My suspicion is that it's a hand ban. I can't know whether the root of it is a false spam report or a blacklisting from an eval.google knucklehead. I suspect it's an eval.google related action.
Not sure if it's related, but at the exact same time, the site in my profile (created for my profile, never promoted), also got a gray bar. Now, if a non-promoted website created just to stick in a WebmasterWorld profile gets graybarred, isn't that just a little weird? Was it an eval.google action gone bad?
Got several directories .. but none of them are Dmoz or SEO purposed. No problems so far .. infact been seeing a decent increase in traffic since Bourbon.
I suspect you've been caught by some dupe filters.
In fact my directory has listings from dmoz and yahoo directory as well as its own unique listings. Assuming it's been filterised, as when I search for "www.mydomain.com" I see "showing results 1-4 from 330.000"), the 330.000 are my directory cached pages that were dropped from google index, assuming so why all site subdomains got PRed zero?
http://dmoz.org/Computers/Internet/Searching/Directories/As far as I can tell its been PR0ed for a while.
My directory is not listed there. All directories listed there arent penalized.
Also I dont know why I suspect alexa being PR0 is related somehow to the topic. But at least only their www subdomain was PRed zero, not all subdomains as it's the case for me.
If they are, I think that they better differentiate between "scraper" directories and human edited sites like my directory is. I have been compiling and publishing my industry links since early 1997 before there was a Google and before I had competitors, because I helped create my niche market sector by being one of the first in the industry, and definitely the first to promote the entire industry without bias by providing FREE links to all of my direct competitors.
Since that time (moving forward as the buzzwords go), I have been publishing updated versions of my original information as an "independent enthusiast" web site. By far I have the most comprehensive directory on this IMPORTANT topic (good thing the United Nations had the site bookmarked - no political comments please), and it's relevancy always helped it to be among the top in Google's SERPs, until early this morning. Most everyone worldwide has need some time or another to reference this information I provide, yet Google seems to be deliberately choosing to CENSOR the information in preference to some of THE WORSE scarper sites I have ever witnessed, along with a few, good industry web sites.
I have written to Google for explanation, and waiting for reply. Apparently it's not because of Google AdSense TOS violation (if indeed Google itself did the deleting), as AdSense ads still appear on the pages and clicks are being tallied from visitors finding the pages from other search engines. However, I only now obtain 10% of the traffic I had before being completely deleted from Google's data base.
martinibuster - I have similar suspicions that you have: I am now pondering if the "nightmares" that some were speculating about has finally began to crop up... jealous/unscrupulous web masters turning in competing sites as TOS violators or Spam scraper sites, and Google employees not familiar enough with the topic and its historic development to make an accurate relevancy determination in the few seconds that they take to judge a web site, deletes good factual content because of the disparaging words of a competitor or someone else with an axe to grind.
Nobody is safe from worry now (believe me I had one of THE most RELEVANT, UNBIASED web sites in a highly competitive sector)
Guess I should start making scraper web sites? They seem to be still in Google's SERPS in my sector.... That's the lesson I am learning from this example.
What say you Google reps?
On the other hand we have a few other sites with directories, but they are not banned. What I think is the following:
1. Google gone mad with "scrapers"
2. They decided to ban sites where directory is the majority of site content pages, assuming that such site would be a scraper
3. They don't care much about casualties, because they are big, kind Google that brings traffic in your home for free and you have to be happy.
On the other hand of the story:
1. Lots of real scrapers are still in, and its totally irrelevant to try to determine them automatically or basing on reports of possible competitors.
2. Simple case (ours actually) - site offers 'widgets' and has normal 10-20 pages devoded to widgets, plus forum, and secured area. Additionally the site has a small directory - 5-10K sites - also devoded to 'widgetry, widget components and supplementary services'. The directory would have MUCH more pages than the main site.
3. Once the site get booted with no way back, no clue, no chance and NO PRIOR WINK that anything it does is against any TOS, lots of people may suffer dramatically, and Google IS responsible, just as when you're happened to become a dominant internet power, you should care about the internet and people who are part of it.
[dmoz.org...]
Many don't even appear!
Can someone please define what a scraper site is?
>the site in my profile (created for my profile, never promoted), also got a gray bar.
From here (Italy) your site shows a Page Rank of 3!
Yesterday I was a PR6 today nothing. Needless to say the traffic has slowed to a trickle - so much for the past 8 years :-(
These scraped linked are less than 10% of my total pages. I have another 150,000 pages of quality newsgroup discussions.
"Scraped" content does not make up a large part of my site... what else could be causing this?
Well at least hopefully I will get a definative answer to what happened, and hopefully they can/I can fix it... otherwise looks like I'm almost back at square one as far as AdSense earnings go, after over a year's dedication to publishing the most relevant information about my niche as possible.
I hope it's just something stupid and easily fixed!
"OHHHHHHHHH the pain & agony of it all....." Who was saying earning Google AdSense was "so easy!"
Google isn't stupid, and I'm sure that they verify sites that they are asked about, so I immediately filed a re-inclusion request and hopefully they will agree that my site should not have been dropped.
I'll let you know what kind of response I get. Please, anyone else that's going through this same situation, post your experiences here as well.
Yep, it was fixed a week after it happened. So-called "canonical url" issues...
[edited by: martinibuster at 8:42 pm (utc) on July 28, 2005]
I filed a request for review through the same avenue as dataguy, and received the default "your site might not have actually been dropped, and here's our webmaster information" email. I responded to the email with my complaint (begging slobbering crying request for reinclusion!) and I'll let you know when (if?) they respond.
No pages, no cache, no links and no PR.
Big Hands came in and killed the entire domain.
RewriteEngine ON
RewriteCond %{HTTP_HOST} ^example.com
RewriteRule (.*) http://www.example.com/$1 [R=301,L]
It works great, and forced the "www" on to the few links in the search engines that had dropped the "www" I do NOT think this could be the problem...
For further clarification, when I say "my directory," I am speaking about the "best content" part of the web site. There are other sections of the site that has other types of information, including artcles.
However, along with my "directory site," another client's web site that I linked to was also completely deleted from Google's search data, and that particular site could NOT be considered a directory in any manner of thinking, yet it was also deleted at the same time.
The information on that site is complimentary to the front page content of the "primary" deleted site, and there was only one link in the navigational side bar to the other site - nothing I would consider excessive "interlinking." Also, they were not "hidden" links, just straight forward text description navigational links between the web sites with complimentary content. Different hosting servers too...
Finally, on some pages of the deleted "primary" web sites, there were links to my "secondary" web site that has relatively good web presense in Yahoo & MSN, while only minimal presense in Google's SERPs (it wasn't designed for Google). Yet, the secondary web site remains in Google (and the others). If there was a problem between the three sites... why didn't they all become deleted?
Anyway, my point is that the recent complete deletion that some of our sites experienced last night - last few days, might not necessarily be because of filters on Directory web sites. However, that just leaves it to be an even greater mystery.
This is the ONLY "major" change that I made to my Directory site that was deleted... added a 301 redirect using coding someone here recommended:
RewriteEngine ON
RewriteCond %{HTTP_HOST} ^example.com
RewriteRule (.*) http://www.example.com/$1 [R=301,L]
However, the other client's complimentary content web site that I was linking to did NOT have the 301 redirect... so that's why I doubt if that is what cause both sites to be deleted from Google SERPs last night.
This site did have about 85,000 pages indexed by Google, so it is a pretty large site. Is this common in the sites that have been de-listed?
