Just guessing but:
Googlebot can spider banned sites, and save the data. At Google HQ, other software fillets said data. Banned sites are filtered there; they aren't allowed in SERPs.
Then the GoogleBot won't have to have a long 'banned' list - It can follow links to any site; much less resource intensive. If a site gets unbanned, bingo, they've got its data, it's back in immediately.
That doesn't explain, 'though, why some banned sites don't get spidered; mine didn't. I thought that was normal: you're banned, GoogleBot stops coming. Maybe all banned sites are flagged into a 'sin bin' at Google HQ; if a link leads there, don't follow it.
Maybe someone who knows for sure can comment? :)
This is speculation.
PS: You have to make sure it's the data GoogleBots which are visiting, and not Googlebot-Images or MediaPartners-Google
|I got my sites back in in a week by completely removing the 'grey' content; pseudo-directory pages with the links running through a redirect script. If the junk is still there when the engineer comes to visit, I imagine you stay banned. |
Just one problem with that. I don't have any junk and Google won't tell me what the problem is. I have removed an ODP section I had that was relevant to my content. Google has reviewed that section after the removal and didn't look at any other pages during the review, but my site remains banned.
Wow, I'm surprised this would happen, since Dead links shouldn't appear at all IMO for any search to have full value for the User.
|I have a site that was taken down for almost 3 years and it's pages continued to rank in the top three positions. |
Runboy, I would guess that if your remaining pages run foul of their filters, you won't get back in, even if they unban you.
Also, I think they don't give advice on individual sites; it could be used by spammers to gauge how far they can go, and it would use up staff's time.
Another data point/Cry for help.
My niche directory site was several years old, mostly hand edited, about 20-25 pages, white hat and was on an ISP subdomain (not it's own full domain) prior to June 16. May was the first month adsense revenue topped $1k. I had recently added hundreds of pages of relevant content that is widely duplicated and also one page of relevant RSS fed content. I was in the process of moving to a new domain and had created test pages on the new domain and linked to it including a link from the old site.
On June 16 Google search traffic went to near zero overnight, although the pages still appeared in Google and the pagerank was unchanged. I semi-panicked and thinking the old site was banned/sandboxed/whatever, I redirected traffic from the old site to the new site using 301 redirects. This increased the new site traffic a bit, but I still did not get any of the old google search traffic. Yahoo and MSN search traffic is unchanged. I saw no changes in traffic on July 22 or July 28. I had some links to clickbank advertisers that I had forwarded through redirects for tracking purposes. I also had some content on a questionable topic that Google does not allow in TOS but it is quite happy to sell adwords clicks for.
What is most likely the problem? The hundreds of pages of duplicate content that I added? If I remove it will my Google traffic come back eventually?
I think you've answered your own question.
The safe way to get more Adsense traffic is to set up a new site, get links to it, and wait out the sandbox. _Don't link to it from your money-maker_.
The real trick is to mimic a 'normal' site becoming popular over time, in terms of incoming links and page number increases.
I have the impression Google is getting very clever at spotting abnormal behaviour in web sites.
"It's highly unlikely that contract quality evaluators would have that authority."
Can someone tell me if exists any "quality evaluator" on the market not owning a website competitor that sites is evaluating?
Always the same stuff. Competitors evaluating and banning websites...Sometimes for Google's interest, sometimes for own interest.
|Always the same stuff. Competitors evaluating and banning websites...Sometimes for Google's interest, sometimes for own interest. |
Why do you assume that "competitors" are evaluating Web sites for Google? Where's your supporting evidence? Why should we believe that the people who hire quality evaluators at Google are stupid? For that matter, how do we know that you aren't an employee of a Google competitor? :-)
In Msg 545 I described adding hundreds of pages of duplicate content to my site. This was a php script that I thought would not result in indexed web pages, but many have shown up in Google and my Google traffic went to near zero June 16. Can I just delete these pages from my site, or do I need to clear googles cache of them also?
If the latter then should I put noarchive in the robot meta tag and allow google to spider my site for a while before I delete the pages? When I delete them, should I literally remove the pages, change the permissions to prevent access, or change the root folder of the pages to disallow in robots.txt? Or is all of this a waste of time and I should start fresh with a new domain on a new ISP?
Of course all we are Google competitors. Someones more than others (that depends if your website works for adsense or not). All we want take free traffic and convert this traffic into money (same Google does).
Difference is we do not ban Google from our websites, as he does with our websites with no reason.
Anyone working on internet is a competitor of the others. This is a competition as any business.
That's why SE must be a machine (not human), with clear rules and clear explanations of the rules and of the actions (not as Google does) and that's why the SE machines are free to order the listings (basic free speech), but can not be free to evaluate or make public bans or ofenses to any website (as Google does)
|Can I just delete these pages from my site, or do I need to clear googles cache of them also? |
I would delete them and have a custom 404 in place which includes navigation to remaining content. I would not 301 redirect any part of a site that may have been the cause of a penalty/ban to another part of the site.
|Difference is we do not ban Google from our websites, as he does with our websites with no reason. |
I have yet to see Google ban anyone for no reason - whether you believe the reason is fair or not is a different question. If there is nothing wrong with your site and it has dropped out of the index than it's probably do to a technical reason on your end or theirs. I'm sure it will return shortly.
Google deleted your site? Get even!
File your complaint TODAY with the BBB.org
BBB.org complaint below:
1600 Amphitheatre Pkwy
Mountain View, CA, UNITED STATES, 94043
Complaint Type: General Complaint
Date Received: 8/16/2005
Primary Complaint Classification: Delivery Issues
Secondary Complaint Classification: Advertising
Google is a public traded company, and must comply with federal regulations. Google has no right to delete any website from their listings. The challenge has been met in court regarding the sells of Yellow Page Ads in Washington DC. A phone directory company was telling the customers of the yellow page ads, if they did not take an ad, their phone number would not even appear in their directory. The company was owned by a public traded company, therefore these unfair business practices were no more than an attempt at extortion. Google is doing the same thing to force website's to buy their PPC (pay per click) advertising.
Desired Settlement: Other (Requires Explaination)
Desired Settlement Explanation :
Resolution:Return my website
MySite.com to their listings.
Customer Service Rep: by email ,
Product or Service: My Website was deleted after 9
Account Number: Google is a public traded company,
Payment Made: No
Purchase Price: 0
Disputed Amount: 0
Purchase Date: 10/2/1996
Problem Date: 7/22/2005
Complaint Dates: 7/23/2005,7/24/2005,7/25/2005
The Bureau that will handle your Complaint is :
BBB of Santa Clara Valley, Ltd.
700 Empey Way #110
San Jose,CA 95128 -4705
|That's why SE must be a machine (not human), with clear rules and clear explanations of the rules and of the actions |
How does telling spammers how to game the SE improve the quality of search results?
|that's why the SE machines are free to order the listings (basic free speech), but can not be free to evaluate or make public bans or ofenses to any website (as Google does) |
First of all, Google doesn't do "public bans." Second, a federal court has already ruled in the SearchKing v. Google case that Google "has no obligation to rank, or rfrain from ranking....any Web site," and that its rankings are "opinions" which are protected by the First Amendment.
If your site adheres to Google's Webmaster guidelines, why don't you simply file a reinclusion request with Google and explain why you think they've made an error?
>>Google deleted your site? Get even!<<
Thanks for great post and bringing this to our attention.
We might need a lawyer to look at this and give us more guidelines. Any volunteers?
Google Deleted your site?
Compliant to the BBB.org
Think they do not care! How many BBB.org complaints do you want? After you file your complaint send a copy to CNBC TV
>>Google Deleted your site?
Compliant to the BBB.org
Think they do not care! How many BBB.org complaints do you want? After you file your complaint send a copy to CNBC TV
Any more suggestions..folks?
>>If your site adheres to Google's Webmaster guidelines, why don't you simply file a reinclusion request with Google and explain why you think they've made an error?<<
Cīmon EFV. People have been writing reinclusion request since Allegra 2-3 Feb 2005. Read the relevant threads and tell me; How many sites were reindexed following your suggestion?
|Cīmon EFV. People have been writing reinclusion request since Allegra 2-3 Feb 2005. Read the relevant threads and tell me; How many sites were reindexed following your suggestion? |
A number of sites have been reinstated since new filters were applied in late July. (See previous posts on this forum.) If that hadn't happened, I wouldn't have made the suggestion.
Also, let's not forget that true bans (i.e., bans that are applied manually) occur for what Google considers to be good reasons--and Google's reasoning, whether you like it or not, is protected by the First Amendment.
>>Also, let's not forget that true bans (i.e., bans that are applied manually) occur for what Google considers to be good reasons--and Google's reasoning, whether you like it or not, is protected by the First Amendment. <<
Sorry donīt know much about the 1st Amendment. But we are witnessing Google CONTROLING information not ORGANIZING it any more. Through filters, algos, updates, tweaking, reshuffling etc..
Does the 1st Amendment calls for that?
|Sorry donīt know much about the 1st Amendment. But we are witnessing Google CONTROLING information not ORGANIZING it any more. Through filters, algos, updates, tweaking, reshuffling etc.. |
Does the 1st Amendment calls for that?
The only way that arguement could even remotely be valid is if the information was not acessable anyway else. Can a site be found by ANOTHER search engine? Can someone type an URL into their browser to find it? If so, Google's not CONTROLLING anything - they're simply using their "judgement" (however flawed it may be) to evaluate what it considers the importance of that information.
Google has banned my web site and refuses to index my spam. This action on their part is causing me considerable unhappiness and significant financial stress! It is... "
I'm having a little trouble with the wording for the remainder of my complaint. Help! :)
Disclaimer: This is posted for my amusement, hopefully it will amuse a few others as well. It may annoy some folks who don't understand that Google has just as much right as they do, to decide what to include on their web site. So be it.
I tend to think that complaining that way; even if you are successful and reinstated, you will probably be placed on page 500. So in the end, it would still be equivalent to a ban. What I wish; is for there to be some way for someone at google to personally tell you what you did wrong if you were manually banned. I've mentioned this before, but it feels like I've been convicted of a terrible crime and I have no idea what I did. It's one thing to lose ranking but to be completely banned is another thing.
>>If so, Google's not CONTROLLING anything - they're simply using their "judgement" (however flawed it may be) to evaluate what it considers the importance of that information.<<
Here is what I found in the online dictionary, and you tell me which one is Google practicing:
# To put together into an orderly, functional, structured whole.
1. To arrange in a coherent form; systematize: organized her thoughts before speaking.
2. To arrange in a desired pattern or structure: The painting is organized about a young reaper enjoying his noonday rest (William Carlos Williams).
# To arrange systematically for harmonious or united action: organize a strike.
1. To exercise authoritative or dominating influence over; direct. See Synonyms at conduct.
2. To adjust to a requirement; regulate: controlled trading on the stock market; controls the flow of water.
3. To hold in restraint; check: struggled to control my temper.
4. To reduce or prevent the spread of: control insects; controlled the fire by dousing it with water.
1. To verify or regulate (a scientific experiment) by conducting a parallel experiment or by comparing with another standard.
2. To verify (an account, for example) by using a duplicate register for comparison.
|What I wish; is for there to be some way for someone at google to personally tell you what you did wrong if you were manually banned. I've mentioned this before, but it feels like I've been convicted of a terrible crime and I have no idea what I did. |
That's what most of us think, but it seems it's too much to ask for. Yet, nobody so far can tell you if it's a manual ban or an algorithmic one. But even if it's an algorithmic one, and since you submitted your reinclusion request and got your site reviewed by a google engineer, can't s/he contact you personally and tell you what you went wrong?
<edited for clearifications>
[edited by: moftary at 3:43 pm (utc) on Aug. 16, 2005]
All this talk is utter nonsense. Move on...geez
How do you think you are going to force Google or any company to spend money crawling your site, pay for storage of your site on their equipment, and also to rank/reward your site while presenting it to the public for free?
In the case of duplicate content I would like to ask anyone here complaining if they will store 1000's of copies of ODP, Feeds, and other duplicate content for me on there web servers and present it to the public for free
c'mon and use your brains.
|But even if it's an algorithmic one, and since you submitted your reinclusion request and got your site reviewed by a google engineer, can't s/he contact you personally and tell you what you went wrong? |
It's really a simple process:
Step 1. - Login to your site via ftp
Step 2. - Delete all duplicate content
Step 3. - Open your browser to your site
Step 4. - If all navigation returns a 404 besides your "contact" page and "about" page you can get a pretty good idea of what the engineers are presented with - is it worth listing, no, not in most cases unless you have a very content rich contact/about page.
>>The only way that arguement could even remotely be valid is if the information was not acessable anyway else. Can a site be found by ANOTHER search engine? Can someone type an URL into their browser to find it?<<
Wish to reply to this part separately.
Google has a search market share of around 40%. That means 40% of searchers will be denied access to sites which Google deindexed!
Lets assume a country population of 275.000.000 . And assume only 30% are connected to the net, which makes 82.500.000 people.
40% of that figure which is around 33.000.000 people are denied access to sites which Google deindexed.
Does that what the 1st Amendment call for?
Not being from the US I can only offer my understanding:
The 1st Amendment gives you a right to speak, BUT NOT the right to force a publisher to broadcast your words at their expense.
>>The only way that arguement could even remotely be valid is if the information was not acessable anyway else. Can a site be found by ANOTHER search engine?
MySite.com since 10-02-96
My KeyWords: MySite.com is #14 Yahoo / MSN # 11 and Google Deleted my site! I will need to be a Google customer! Forced to PPC
Kicking up a stink with Google is probably the _worst_ idea I've read here. I paraphrase from memory one poster here who organised protest emails from his site users, got back in, and wrote 'my site's back in, but my PR is 0, is that normal?'.
I deleted any and all dodgy content on my sites. Wrote a very polite and humble re-inclusion request. I got back in in a week, same PR, same SERP positions.
I think the 'damn Google' people just don't want to change their sites, and hope that by getting the unwise to join in their clamour, they can get a mighty corporation to spend valuable man hours dealing with their guff.
-> I mean, if your site get banned,
and you don't make serious changes,
and _even_ if it's then manually unbanned (unlikely),
isn't it going to trip the same filter again the next time the bot comes by?
Hee hee, if one was evil one could encourage this folly, "yeah, go for it, run up and kick the big bully", except innocents might get their lives and livelihoods ruined by following such 'advice'.
>> Does that what the 1st Amendment call for?
What a silly comment - the 1st Amendment, as far as I understand it, would only really be relevant if Google were a publicly owned entity, when in fact it is a privately owned business that can do whatever it damn pleases. We should just feel lucky that it has done so much good for so many of our businesses, not start taking that for granted and whinging.
Just my 2p worth!