Temporarily Banning Googlebot - Last Resort to Recover a Flagship

[JimLahey]

Had too many useless canned responses and frustration trying to determine if my site is penalized or simply caught in a technical glitch....the April 26th curse. I am banning Google from my site via robots.txt for a month. My hope is that if what occured on April. 26th was a glitch I can get reindexed with a clean slate. Anyone ever experimented with such a move...robots.txt only and not the Google removal tool (guaranteed looong time frame).

User-agent: *
Disallow: /cgi-bin/

User-Agent: Googlebot
Disallow: /

User-Agent: Googlebot-Image
Disallow: /

User-Agent: Mediapartners-Google*
Disallow: /

[Iguana]

Funnily enough I had a sticky mail about this from another member this morning. I did this last year:

[webmasterworld.com...]

I noted in the thread that my main site (the one I used to link to a lot from the site I banned Googlebot from) recovered. It disappeared again a few months later so I don't think there was a cause and effect.

From what I remember, it took about 6-8 weeks before the whole site went supplemental. Three pages were left in the index because of DMOZ entries. The home page began to rank #1 for it's own name during this period (again I suspect this was not related to the Gbot banning). I kept Googlebot banned for about 5/6 months. I then allowed it back, but ever since then we've had Big Daddy and the Supplementals problem. I still don't have all the pages indexed - but I do have rankings for the pages that are back. it has not fully recovered it's rankings but it does feature on the first page of some searches.

So, I'm really not sure how effective it was - it was worth doing for me because I had about 2 visits from Google a day by the end. Certainly it's not worth doing it for one month.

During this time, my blog about the same subject (and named after the site) came out of the sandbox. That now has the same sort of power as the old site did. But because it's on Blogger, it's quite difficult to insert pages in the way I would want to. Maybe I should have concentrated on moving the content into the blog? Google loves it.

[Sunnyvale]

Iguana - I get the impression you do not think 1 month is long enough to clear the slate. Do you think the 5 to 6 months you did could be reduced to say 2 to 3 months with the same result.

[Iguana]

I think it depends on the State of Google and the PR and size of the site.

If Google settles down and stops making whole site supplemental and starts indexing more, then maybe things could be shorter. My site has a PR of 5 and about 400 pages and Google is being very slow at getting all the pages back in the index. A bigger PR would no doubt help.

I could have removed the Googlebot ban after about 8 weeks and it had gone supplemental. Recovery (before the Big Daddy ongoing debacel) might have been 2 months.