Welcome to WebmasterWorld Guest from 220.127.116.11
If you run a web directory, feel free to post your experience here.
If I was an evil virtual dictator on a power trip, and I wanted to keep my subjects guessing, I'd not implement one change, I'd implement two at the same time.
When a site owner is allowed to fix up their listing in a directory, they must be quite likely to put the exact same text in any directory they list under. Could trigger a dupe penalty for a phrase or two if I'm reading the Google patent right. Perhaps too many dupe phrase penalties on one site nukes it?
It might be that anything with the word 'directory' in a meta tag is more likely to be on top of the serps when a webmaster looking for directories to submit to is, well, looking.
Hope that helps.
But Google was that name that caught my attention... as in "great, just what we need another search engine."
However, after seeing them a few weeks I finally gave their beta search engine a try myself, and "WOW!" This Google is GOOD!
Now after all their years of growth and most always featuring my "stuff" near the tops of appropriate search results, they completely delete my information from their data base.
"Where's the love & idealism we once shared in our youths, Google?" I bet if Larry or Segey discovered what was happening to me (moreover my information), some ears would be ringing at the Plex! Too bad they don't get in the trenches anymore... that's why Google has become just another soulless corporation IMHO.
Two OTHER sites are EXACTLY the same. PR 4 on both. Small link directory (< 100 links). Many pages of nearly identical content and google api links at bottom of page. Same code, even the same language in many areas... not banned.
None of the sites was a directory site! They all have recip link pages...
Here are the only differences I can see between these sites.
- Banned site had more traffic
- Banned site had more backlinks
- Banned site had more pages indexed by google (way more)
- Banned site was overly SEO optimized on the home page
If this was a simple algorithm flip of the switch, then why didn't the other two (lower traffic, lower indexed) sites get banned as well? Could it be that the algorithm has a pain threshold for amount of scraped/recip content?
The weird thing here is that my banned site saw more googlebot traffic this month than any month in it's history... why would they re-index almost my entire site and then boot it?
BUT, the similarity is that I noticed MUCHO googlebot slurping, even though the site was already completely deleted from their SERPs...
And the mystery keeps getting deeper...
Same thing happened to me, well, actually not the same but very similar... This month, Google sent more traffic to my site than ever.
Question for those who were banned: We run some strictly opt-in newsletters that are sent out to thousands of subscribers. Do you have any kind of newsletters? Is it possible that google finds out somehow these newsletters and classify them as a spam?
("gosh darn" that is depressing - the term "banned" being associate with my site).
In fact, that was always something that I was considering to help increase return visitors... most find the information that they need and I don't see them again (except for those in the industry constantly "spying")
However... now my site has been actually banned by Google, so what do I know anymore? I went from "riches to rags" over-night....
Therefore I must be a bum, according to Google's treatment...
"Leper... Leper... Stay away from me or YOU will become infected and thrown out of the Google search engine community too!"
Ok, so those that feel like they were executed – could you answer the following to see what they had in common?
1. Dynamic or static pages?
2. How long were they online?
3. How many pages?
4. How were they originally seeded?
5. Were they free inclusion or paid listings?
6. Were they reciprocal link directories?
7. Template based – what changed in your template besides <title> tags and listings when going through the categories?
8. How many listings per page?
9. Was there other content besides the directory listings?
10. Did you offer/use ROS links?
11. What was the PR before they seem to have been penalized?
12. If you used an off-the-shelf script did you change the default paths etc?
The Contractor has just given the guidelines of running a successful directory. The only points I would add would be -
13. Does a related:www.MyDirectory.com search result in other directories you own?
14. Have you been making a lot of noice/promo about your directory?
I still don't hear anyone reporting on obvious known scraper sites. Have they been banned at the same proportion (roughly 25%) as the sites discussed in this thread? I can't be of help there because I don't watch for scraper sites, I barely watch my own sites in the serps.
My heart goes out to those of us who have lost our primary income with this filter, but my experience is that Google has been as reasonable as I could expect in these situations. Typically after they tighten up their filters they seem to manually adjust and eventually the good guys get their listings back. My advice is to not make major changes to your site, unless your site was overly optimized for search engines and under-optimized for human visitors. Now is the time to work on making your sites as useful as you can to human visitors... it will pay off eventually, if not through Google, through their competitors.
I've been at this before Google started and long before AdSense and I can survive without Google, though it would be tough. Fortunately the one site of mine that has been dumped is a minor site, and the loss in traffic has resulted in only a 20% reduction in AdSense revenue for the first day for that site. I would say that I feel personally insulted more than anything, though I know this filter has hurt others in more tangible ways.
Typically after they tighten up their filters they seem to manually adjust and eventually the good guys get their listings back.
This seems true for the ordinary Google updates. After an update there are always many reports on this board of people who went from page #1 to #100, but after some weeks when the algorithm is tuned many come back slowely at reasonable positions.
The difference now is, that most people report a complete removal from the index, not a drop in the SERPs. This would indicate a manual ban which is much more difficult to reverse than a different position in the SERPs. It would have been far easier when they just removed rankings from the sites but let them stay in the index. In that case people here would just call it an update. There must be a reason that they now chose to perform a complete removal. I can only think of two reasons:
Even if the sites have gone on all of the dcs at once - this suggests that Google still has a way of wiping the datacenters of sites in one go - so there may be a master list of domains/canonical urls that might have a technical glitch. However, it does look like bans to me.
Recip links are not ILLEGAL according to Google. Using Findwhat as a PPC search engine is not ILLEGAL according to Google. Since 1/2004, the PPC engine receives around 15K page views per day. All told, the domain earned around $1,000 per month and NOT from Adsense.
Lucky for me $1,000 a month is a very small portion of my total earnings. I can imagine many guys landing in hot water over this. Bottom line, if you're banning people for a reason: TELL THEM WHY.
It seems Google affected an entire "class" of website owners. They did so with malice and forethought and 0 accountability. Google has made the whole internet marketing business unpleasant with these violent upheavals. I just recovered from the $1,500 per month they've taken away by turning off 4 content sub-domains that were squeaky clean and now this.
"The weird thing here is that my banned site saw more googlebot traffic this month than any month in it's history... why would they re-index almost my entire site and then boot it?"
Same thing happened to me, well, actually not the same but very similar... This month, Google sent more traffic to my site than ever.
Same here. Actually two days before my directory was banned, it hits its top traffic day since it started.
There is a technical issue/problem on Google's side which caused some domains to drop temporarily. If this is the case however, the ban should be visible on some data centers only, not on all simultaneously. Did people check if their site disappeared on all Google DC's?
That's a primary thing to do. And yes, banned sites were dropped from all Googles DCs.
"then"? you mean "than" I recon. And no, English is not my 1st nor 2nd language, and still I can write/speak it. What about you, do you know Arabic?
Let's not turn this thread into a childish flame war please.
I seriously was not trying to (did you read the sentence after that). I remember GG looking at someone's site and bringing them to the carpet on several spelling errors. And BTW "than" is the correct usage and I know that…hehe
I dont have a definition for a scraper site actually, and I dont care to have one
You don't have to have one as in this case we are discussing what Google's definition may be. How often do you see dogpile's results ranking in the serps? I never have. Again, a directory and a search engine are two different things.
The above is not off topic in any way, shape, or form.
JuniorOptimizer msg #:73
Recip links are not ILLEGAL according to Google. Using Findwhat as a PPC search engine is not ILLEGAL according to Google.
If you believe reciprocal linking (high % of your backlinks) and also using someone else's content exclusively for your site content is a good/long term solution to run your site I believe most SE's wouldn't agree with that (only my opinion of course). If people are waiting for Google or any other SE to come right out and post 1001 things not to do they will be waiting a long time.
You must be a very dirty webmaster to assume that everyone is as guilty as you. Many, many innocent webmasters are affected by Google all the time.
Googlebot is still visiting my site, I get many requests from: crawl-66-249-65-171.googlebot.com.
Weird! Maybe you are not banned really and you are very stupid that you thought you got banned when you havnt been (according to google automated reply)? :P
Now, seriously, this might give you a chance to clear any trace to the term "directory" in title and meta tags, and waiting for Gbot to reindex you after making your prayers.
I for one, according to awstats, never hit from gbot since the ban exactly 34 hours ago. Yet, I took my advice and did so, but out of being disperate.
The Contractor has just given the guidelines of running a successful directoryMaybe i am being thick but i don't see the guidelines - merely questions?
Ofcourse they are not guidelines if you read them by the letter. But those questions guide you along the lines that one should be thinking about while building a directory.
Added: Inspite doing their best, I sincerely feel few may have faced the axe, which might be nothing more than a collateral damage, and I entirely empathize with them. My directory site may still has not been axed, which I would like to think due to all the right things I have done with the site. I still may face an axe, as MB says by one of those eval.google.com knuckle heads.
[edited by: McMohan at 11:32 am (utc) on July 29, 2005]
I'm not sure why you keep posting your inane commentary in this thread. 0 % of my content on my network of sites is scraped…..The main reason I asked for recips or payments is because if you offer free submissions you get spammed to death. You get it?… You must be a very dirty webmaster to assume that everyone is as guilty as you.
What's your problem? I am not accusing anyone of anything. I never said anything about paid inclusion – have I? If you can't look at your site and tell what you have done wrong, how am I going to not look at your site and tell you? If you would rather whine, attack, and come up with conspiracy theories then to try to find the reasons why your site may have been booted so be it. Dirty webmaster…nah, ask anyone that knows me. Keep whining, attacking, and coming up with conspiracy theories …
Typing in 400 category pages by hand is a lot of work. I also wrote the software script. I'm still on pace for $180,000 this year, so I'm not really crying all that bad.
Just so you know: Scrapers actually "scrape" content and then monetize the traffic through AdSense. That's what Googleguy said they went after on July 28th. Plus why exactly did they nuke the entire domain? Two sub-domains on two seperate IP addresses. And I'm supposed to "keep looking" for what may have possible violate their incredibly vague guidelines?
BTW, there are still many scrapers out there.
Exactly. I am not going to question Google intenetions, I don't care about WHY they do things - although MONOPOLY in anything does meansresponsibility. the only thing I want is to hear that YES, WE DON'T WANT DIRECTORIES TO STAY IN. Then I wouldn't ask why Alexa is still in, or Yahoo, or DMOZ - I'd let big guys hug & kiss each other, and I would immediately remove directory - the result of hard work for years, even though its human-edited. Just to please Google, because they are monopoly and we have to obey :(
BUT COME AND SAY THIS, Google(Co.)!
p.s. Receive a message for potential cutomer today, telling that our competiros say that "they are not even listed in Google". What this actually means is that this ban directly leads to financial losses to us. Since we have nothing against Google webmaster guadilines or TOS, I am cosidering filing legal complain, this monopoly and no dialog starm making us mad. And G is totally wrong if they think that they're too big to bother.
I am cosidering filing legal complain
Good luck. The Google search engine is a free service and judges have ruled in the past that their SERPs must be interpreted as "Google's opinion on the web". Like humans they may change their opinion every time, and deleting sites from the index is just one way to express their opinion.
Just so you know: Scrapers actually "scrape" content and then monetize the traffic through AdSense. That's what Googleguy said they went after on July 28th.
Look, you can agree or disagree with what I am saying, doesn't matter to me. GoogleGuy never said anything about going after sites that scrape for AdSense money so let's keep things factual instead of more theories. Here is the thread [webmasterworld.com...] and you will not see the word AdSense in his post at all (msg#4). I have seen many sites/scripts that have FindWhat/Searchfeed fill and they are usually nothing more than an endless supply of search terms in one column and backfill PPC listings in another. If you call that a directory that is fine, but to me it is not (and by definition it is not). I am not saying that is what you have done, but simply your site may include some of the same qualities or scripting that make it appear this way.
Plus why exactly did they nuke the entire domain?
That is what this thread started out to be. What qualities do those nuked sites have in common or what qualities do they have to make them look like scraped, generated, or low quality sites to Google. It seems no-one really wants to figure that out….
Ok - some of the side effects of the above - normal directories will get hit (even ones with unique user submitted listing - the user submitting probably does not vary the text to much between directories), aswell as sites which have a large number of seemingly outbound links as page content.