|Does Google Ban or Filter Web Directories?|
I think the subject worth a thread itself. It's a suspision so far. Yet I don't see dmoz, yahoo nor any major web directory were banned/filter nor PRed zero as my web directory did. I tried to check it in Alexa (powered by google) and I see some results from my site. Appearently, Alexa brings old results from Google but something weird is that Alexa itself has PR0 now. But that's another story!
If you run a web directory, feel free to post your experience here.
EFV - we are not talking about losing places in the SERPs - we are talking about a total ban of the sites in the index. That is what censorship is - there is absolutely no way that a visitor has a chance to view a site via Google that was caught in the July 28 deletion.
I agree that people use search engines to determine relevancy - but Google has always used a "democratic" approach before this widespread deletion of sites. A site can (theoretically) start near the bottom of the SERPs, and as people find it and like it - and correspondingly link to it, it gradually makes its way up the ladder (paraphrased straight from Brin and Page). This can not happen if a domain is completely deleted from the index.
My site went from 10,000 refs a day from Google to absolutely ZERO. I'm not a scraper, in the ODP since 1998, permanent "original" Yahoo directory member, and 5 years of good quality, non-reciprocal links from authority sites - many PR 7 or higher. Top of the SERPs one day, then the next day (July 28) every page of my site is deleted. If that isn't censorship, I don't know what is.
An Analogy - The people voted in democratic elections, I came out on top for my key targeted search terms, and BOOM - I was "overthrown and executed" by a Google cleansing - LOL. Friendly fire or not, I'm still completely out of commision.
For what its worth and a faint glimmer of hope - I did get a response to a reinclusion request that was not a "form" reply (they included a greeting with my name that I only put in the text of the message) - they said they would forward my e-mail and website address to "the Google search engineers for evaluation"
Page 25 and we're still explaining the difference between being surpressed in the SERPS and having your entire domain de-indexed.
GoogleGuy has been really helpful in this thread. Today I received a few referrals from ODP clones that run AdSense. So the idea that ODP clones and "scrapers" have been targeted appears to be erroneous.
All right. My three sites had autogenerated pages (by my own script) with autogenerated links sections, a randomised, relevant RSS feed and randomised quotes.
The links were produced by a well known site generator, which I then customised away from the template. I wanted the links only.
Two satellite sites had a lot of these pages (about 1000 each). The main one had 100. All sites interlinked. Keyword lists were got from Wordtracker, Overture, and a site offering a list of most lucrative Adsense words, which I edited heavily to find ones relevant to my own products.
The only thing the sites had in common was these type of pages, the site interlinking, and that the pages had lists of links in them.
I'd thought they looked OK, and were relevant to what searchers were looking for, but putting myself in a seasoned human reviewers shoes, like Yahoo, probably not [sigh].
I'd guess Google is going after the Adsense spam-style sites. You know the ones: big fat Adsense block at the top of the page, rubbish links and a free reprint article as the main text.
Anyone else caught is collateral damage.
I requested reinclusion and received a templated response a few hours later from help@google. If my site was manually removed (by eval.google) and not removed by an algo or filter change does Google clearly tell me that in its response to my reinclusion request?
I highly suspect now that it's not a matter of ODP clonning or anything. The algorithm IMO catches sites with outbound links greater than a certain value, or maybe has a percentage of outbound links to internal links greater than a certain percentage. Maybe?
If that's the case, a simple redirect link, like "ra.cgi?" that WW uses would solve the problem.
|does Google clearly tell me that in its response to my reinclusion request? |
No, whether approved or disapprove you dont get a responce. Search for your site daily for a month :)
|Today I received a few referrals from ODP clones that run AdSense. So the idea that ODP clones and "scrapers" have been targeted appears to be erroneous. |
Me, myself and I have been saying that for ages here. I have three ODP clones never been affected by google filteration.
>>>If that's the case, a simple redirect link, like "ra.cgi?" that WW uses would solve the problem.
I use a script like this which is blocked by robots.txt in my forums, just incase spammers post something that isn't caught by moderators and then picked up by Google. This is meant to prevent linking-to-bad-neighbourhoods penalties.
I can effectively dispell this theory.
Just an update.
In my case Google Help was very quick (yesterday) to respond to my mail and they wrote back and said they have forwarded the mail to their engineering team for further investigation.
Has anybody ever recieved a response from the engineering team?
Time runs. Google answer just automated responses to many of our request.
We are hundreds, probably thousends at same situation, and as I said at previous posts, for me will be same if they restore my site to previous status. For me is unacceptable this kind of policies. Probably we will have not better moment for start a real supported movement against a company becoming a dictator in Internet.
All webmasters with deleted websites posting here, still having tons of visitors every day (my banned directory still having more than 10.000 unique visitors a day). Is just organize the movement and let the wave grows. Someone agree?
|I can effectively dispell this theory. |
You mean that you already use a redirect script for your outbound links, and was banned?
Ok, thirty eight pages with no conclusions, so please add your thought to the following some-proven-wrong theories..
1- Google ban/filter directories: No, some directories are doing well in serps (sevenseek, for an example)
2- Google ban/filter ODP clones: No, I have three of those and they are doing well in serps. Others stated the same.
3- Google ban/filter according to the term "directory" being in title, meta tags, etc..: No.
4- Google ban/filter directories with a high percantage of outbound links: No.
5- a) Google does not ban nor filter according to an algorithm, it filtered its serps according to editors: No, all directories were nuked in the same time. b) Probably they were collecting/reviewing some directories domains and then put them in the filteration list at once?
6- They have a technical glitch.
7- They ban ODP clones with "computer directory" only. <- my first bet.
8- They ban directories with a sitemap with the same ODP structure regardless than the content. <- my second bet.
Please share your theories no matter how silly they are. Please do not post long stories that are off topic. Please do not go into silly arguments that does not add a thing. Please do not drive this thread out of its purpose, which is its subject that was never answered so far.
Whether I am unbanned or not, and knowing that I will not receive a response from google engineers in either cases (yes, I have a past experience with such), this thread should be ended when its subject question gets answered.
Moftary you are skewing the facts and potentially misleading people reading this thread with comments like below:
|I have three ODP clones never been affected by google filteration. |
You may have been able to make that statement a week ago, but you cannot make that statement as fact now.
I have personally built and developed dozens of directories for myself and other people. How come not one of those is banned? I can answer it very easily.
They have completely original structure and content, taking weeks, months, and years of development.
Not one of them includes the structure of any other directory.
Not one of them runs their outbound links through a script, url masker, or click counter.
Not one of them contains titles/descriptions that can be found anywhere else (unless someone has scraped/copied them).
The above is your answer. You may not like it, you may not think it's fair, and you may not agree with spending that much time and effort on creating/developing something, but those are the facts.
Please get out of the denial stage! The ODP clone sites with 80% of your page space taken up by AdSense ads is nothing unique and is nothing of value to anyone. You have disregarded what many have already told you, what GoogleGuy stated, and what would be quite obvious to anyone else. Unless you are counting "in your face - taking up more page space than everything else combined" AdSense ads as "value added" content – they have nothing.
I shouldn't even be responding to this thread, it's just I certainly don't want people reading your conflicting posts and damaging their own site. People complain about scraper sites and how they should be removed because they offer nothing unique or of value. What makes a ODP clone any different when judged on content?
>> I have three ODP clones never been affected by google filteration.
don't celebrate or kill yoruself, just yet. This is google's first, major purge. They will refine to put back some innocent ones and catch others.
|You may have been able to make that statement a week ago, but you cannot make that statement as fact now. |
Elaborate please. I have three ODP clones that are PR5 and doing well in serps. How come they haven't been filtered? Please answer this question :)
|I have personally built and developed dozens of directories for myself and other people. How come not one of those is banned? |
Kindly read other people posts that state that their directories, with unique submissions, were nuked although has nothing to do with ODP.
|Not one of them runs their outbound links through a script, url masker, or click counter. |
I said MAYBE BECAUSE WE ARE NOT USING A REDIRECT SCRIPT WE WERE NUKED. That means that we simply dont do that (out of topic then).
|The ODP clone sites with 80% of your page space taken up by AdSense ads is nothing unique and is nothing of value to anyone. |
Here we go starting a silly argument again. Wearing the clothes of a judge and assuming that my directory is nothing but an ODP clone although it has a one year old seed of ODP and yahoo directory, with thousands of unique submissions. I dont know even whether you know my directory url or not!
|You have disregarded what many have already told you, what GoogleGuy stated |
I repeat, GG has said clearly "with no value added", of course you are saying that we, all the banned directories owners, has no value added but adsense and I dont know really on what basis is your assumption. Google directory value added service is the PR. Alexa directory value added service is the ranking position, info, etc... My directory value added is the new thousands of unique submissions.
You dont have to comment on anything, nor even defending your own theory. But please, for god sake, answer the first quote above with a direct answer.
|>> I have three ODP clones never been affected by google filteration. |
don't celebrate or kill yoruself, just yet. This is google's first, major purge. They will refine to put back some innocent ones and catch others.
I thought so, and eagerly anticipating that they catch the others. Then and only then the ODP theory would be true.
It's been a week so far and nothing changed. My ODP clones are standing and my unique directory, seeded by ODP and yahoo, is purged.
You think that they would tweak the algorithm (I dont know what algorithm that misses a typical clone and catch a modified clone), I really hope so. That would bring my big directory into life, and send my sins to death and most importantly that would explain the myth and put my mind into a relief.
OK, I will answer your questions.
|Elaborate please. I have three ODP clones that are PR5 and doing well in serps. How come they haven't been filtered? |
The probable answer to your question has been answered by walkman in msg#374 and also in [webmasterworld.com...] msg #:16 " It's something that we're going to look more at over time, too."
Now after reading msg #:4 in the same thread, I would imagine this will be an ongoing thing rolled out over time.
|Kindly read other people posts that state that their directories, with unique submissions, were nuked although has nothing to do with ODP. |
Agree completely. From the those that stickied me it looks like scrapers, full/partial ODP, copied/mirrored newsgroups, having multiple sites using the same content, or using content taken from elsewhere as being the majority of their pages/content. I have yet to see one site that has been hit by this that does not have one or more of the above conditions. Which all boils down to the majority of each of these sites were made up of duplicate content.
|Here we go starting a silly argument again. Wearing the clothes of a judge and assuming that my directory is nothing but an ODP clone although it has a one year old seed of ODP and yahoo directory, with thousands of unique submissions. I dont know even whether you know my directory url or not! |
I know one of yours that's banned and it completely fits the description I gave. I'll give you credit on the others…you make them harder to find when you use your nameservers as the domain names. How did your sale go of that Russian domain? Amazing what you can find out if you look.
|I repeat, GG has said clearly "with no value added", of course you are saying that we, all the banned directories owners… |
I never said such a thing. I have clearly stated that every one I saw (including yours) contained one or more elements that I have described over and over and over again.
Do you really think that .1% or less of original content on top of an ODP clone along with 80% of page space taken up by AdSense ads defines the term "value added"?
For anyone that had their site banned, did anyone have LESS than 1000 link partners (either reciprocal or not)
What I meant by 1K link partners is 1000 Outgoing Links (link partners, hand picked links, directory, whatever)
Every site in MY dataset that got banned has >1000 Outgoing Links.
Almost All scrapers have more than 1000 Outgoing Links.
My banned sites fit the Contractor's description. They also had a redirect script in the links. No ODP, text writen by me, but lots of outgoing links, and what I now realise was probably duplicate content. And they were doorway pages too, by Google's guidelines, which now have a horrid relevancy to me.
And: Now I think on it, the 100 grey HTML pages used on my main [banned] site would be a significant proportion of the HTML pages on that site (the rest would be Php bulletin boards and the like).
And: _the redirect script ran from the main site_. It was used on the main site, _and_ by the two other banned sites.
And: The redirect script forbids off-site usage. The GoogleBot would just get an error message when it tried to follow the link from their database.
[Laughs ruefully] Done down by my own mania!
Hope that helps someone.
I stickied my URLs to only two members, how - magically - these URLs reached you?
hmm, I smell high traison on the WW community.
BTW, WW should give the one that started a thread the authority to close it. If ODP clonning is the answer that everyone agrees about here but me, then subject is closed.
|I thought so, and eagerly anticipating that they catch the others. Then and only then the ODP theory would be true. |
Again, you only picked out a small portion of what was said. It's not only ODP clones. I won't repeat myself for the 20th time.
|I stickied my URLs to only two members, how - magically - these URLs reached you? |
hmm, I smell high traison on the WW community.
Nope, no-one stickied me your site. You leave tracks just like your websites do. It doesn't take a wizard to track down what sites people have when they post it on other forums. Once you find even a small bit of info you can map out almost anyone unless they are really, really good. I found quite a few of your sites, your different nics, what hosting you use/offer, projects you've worked on, and BTW another one of your directories may be banned the one with "it" in the domain name.
|BTW, WW should give the one that started a thread the authority to close it. |
You can keep it going as long as you want, nothing new can be added.
The obvious has been stated, the facts have been stated, and the conspiracy theories have been stated. You can choose to believe one of the three, but unless you can prove otherwise - don't misguide others in your process of denial.
Contractor, you're killing me, these posts are just too funny. And for others who might be reading this thread, pay attention to what he's saying, it can't be said much more clearly. Have to put these posting among my all time WebmasterWorld google forum favorites, LOL..
Google To Face Class Action Lawsuit:
Take a look at this!
|Has anybody ever recieved a response from the engineering team? |
Actually, yes, via the support group. One of my sites was dropped once because of a bug and the engineering group found the problem and fixed it. I was glad I'd written as I don't know if they would have been aware of the bug if I had not sent in a couple of emails about it.
I like most this part of the PR regarding " Google To Face Class Action Lawsuit":
"NOTICE: Google.com Doe's Not Have The Right To Exclude Any Website Form It's Search Engine Pages! Listen to that "Search Engine". By the statements made by Google.com. Google.com is a search engine. Yahoo is a directory and can add and remove anything from their directory listing. Yahoo can choose to make you number one for a payment in it's organic search.
Google.com was promoted by all of the webmaster's that it has now deleted, that Google.com would be a fair and impartial results. Be a good webmaster, create a lot of content and work all day and night begging other sites to link to you. So our investors can get rich and Google.com can just decide to destroy, your hopes and dreams, that you have given everything to have your website on the Internet!"
"Google To Face Class Action Lawsuit"
I'd be embarrassed if I were the one who wrote THAT press release - with all the typos, misspellings and poor choice of wording (a press release should be given much more editorial diligence than a quick posting on WW), not to mention the absence of clarity of thought. The "document" displays very poor "professionalism" and I would rather STARVE than have someone with such total lack of business acumen represent me and my professional interests - EVEN if I thought about entertaining such nonsense!
|Google To Face Class Action Lawsuit |
Yeah, sure. Maybe you aren't aware of the federal court ruling that said Google's rankings are "opinion" (a.k.a. constitutionally protected free speech).
By the way, I agree that The Contractor deserves the Webmaster World equivalent of an Oscar for great detective work and incisive, informative posts.
|Google To Face Class Action Lawsuit: |
Yeah right, I'll take that seriously. Written by a real legal mind I can tell. There are more spelling/grammar mistakes in that document than in my daughter's 4th grade class.
I'm not trying to belittle anyone, but I will not sit back and watch people claim innocence about their super high-quality one-of-a-kind directory when you see their sites that are ODP clones and you have to scroll and scroll and scroll to the bottom just to see the listings because 80% of the other space is plastered with adsense. The only reason there are any listings is to target the AdSense ads. If someone wants to talk about how wrongly they were banned and how high-quality their site is, then I think the readers of this thread should know the truth of the matter.
I have a thing about directories. Not imported, scraped, duplicated ones – real ones. No, not a directory consisting of 3 pages of "reciprocal links" called a "resource directory". I've been to sites that sell jewelry for instance. Now who in their right mind would have a "resource directory" on a jewelry site that includes everything from diet pills, gambling/casinos, and p#rn? Yeah right, I'll buy my wife's anniversary gift from a reputable place like that….
I think they are great when done properly. There are tons of sites that are in the last 500 results that are good sites, but they would never be found if someone didn't search them out and put them in their directory or the have the site owner add them. I'm not talking about grabbing a script and importing 4 million listings, I'm talking about planning out the structure and adding each listing one-by-one and titling/describing each listing. It's not for everyone and most every one I know thinks I'm nuts when it comes to this subject ;)
>> "Google deletes thousands of sites, pretending to be eliminating bad neighbors
>> Google To Face Class Action Lawsuit:
>> The so-called removal of the negative sites from the Google.com search pages. Have webmaster's crying foul!"
Google killed my neighbor too I guess. Nice little old lady who ALWAYS minded her own business. I was very sad to see the ambulance by her house last night. Shame on you Google ;)
|Google killed my neighbor too I guess. Nice little old lady who ALWAYS minded her own business. |
Hmmm… must have been one of the innocents that got caught up in the collateral damage…...
HA! Maybe this nonsense will kill this thread, finally!