Welcome to WebmasterWorld Guest from 54.211.82.105

Forum Moderators: Robert Charlton & aakk9999 & andy langton & goodroi

Message Too Old, No Replies

I've banned Googlebot

What Will Happen Now?

     
6:25 pm on Jul 16, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 20, 2002
posts:889
votes: 0


Last Saturday 8th July I disallowed Googlebot in my robots.txt for one of my sites. The only effect so far seems to have been that my description in the snippet has reverted to my DMOZ entry and my page count in Google has gone down from 472 to 469.

I expected to see something a bit more radical happen. Can anyone predict what will happen? Will my pages go Supplemental or even disappear from the index?

9:16 am on July 17, 2005 (gmt 0)

Senior Member from DE 

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 20, 2003
posts:878
votes: 5


Your page will be completely removed from the index. A detailed description is given here. [webmasterworld.com]
9:32 am on July 17, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 6, 2005
posts:91
votes: 0


Iguana, wait a little longer, we kicked G off one of ours and it took a week for the site to disappear but then that did conincide with an update.

[edited to correct explanation]

8:09 pm on July 17, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 20, 2002
posts:889
votes: 0


Doc_z

I havent used the Google removal tool and don't intend to - purely robots.txt. Can I still expect to me my complete removal?

8:24 pm on July 17, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 6, 2005
posts:91
votes: 0


Iguana,

We didn't use the removal tool like doc_z, just robots.txt, all our pages disappeared except for the link url into the site from external pages (without text or snippet). So I would expect the same to work for you.

10:54 pm on July 17, 2005 (gmt 0)

New User

10+ Year Member

joined:June 27, 2004
posts:34
votes: 0


hehe... It's time to sell Google's shares... :)
11:24 pm on July 17, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Apr 29, 2004
posts:180
votes: 0


I've stopped calling my friends and when they call I swear at them and hang up. What will happen now?
12:14 pm on July 18, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 20, 2002
posts:889
votes: 0


Thanks ncgimaker, useful to know I can use this rather than a removal

As you say, I may have to wait till an update (another 3 pages less in Google today) for everything to disappear properly.

2:42 pm on July 18, 2005 (gmt 0)

Senior Member from DE 

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 20, 2003
posts:878
votes: 5


I would expect that all information are removed from the index (even URL only entries). The time until all pages are removed might depend on the crawling frequency.
2:46 pm on July 18, 2005 (gmt 0)

Preferred Member

joined:July 8, 2002
posts:584
votes: 0


I'm curious to hear the reasoning behind the ban. Too much traffic? A private site with personal secrets you only want to reveal to a select few?
2:46 pm on July 18, 2005 (gmt 0)

Moderator from GB 

WebmasterWorld Administrator mack is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:June 15, 2001
posts:7617
votes: 21


I realize there has to be a good explination, but I have to admit I am curious. What made you decide to ban Googlebot?

Was Google simply not delivering traffic, was the bot using up to much bandwidth or another reason?

Mack.

9:12 pm on July 18, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 20, 2002
posts:889
votes: 0


The reason for banning Googlebot is pretty simple - it can't hurt. For this particular site I rank between #13 and #70 for my own name(!). Therefore I get about 1-3 visitors per day from Google. I've done all the getting rid of 302's, sorting out non-www issues (not that there was more than 1 link to non-www), emailing GG on the update group.

There's a possibility that Google is looking at related sites whether through history of links or even whois. Maybe by taking this site out of the index the other site that makes a bit of money might return to it's former position. Also maybe my blog on blogspot (named after the removed site) may rank on the first page so that bands looking for someone to review their CDs will find a link to my site. As I say, it can't hurt to try. I don't want to use the remove URL because I may want to remove the ban in a month or so.

The last time I took this action was with with Yahoo about 18 months ago. Within 10 days I was ranking really well (as compared to last on exact string searches). Okay, I know it was just coincidence then, but when logic fails, choose the irrational.

10:27 pm on July 18, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Jan 3, 2004
posts:315
votes: 34


I blocked one of my sites using robots.txt thinking that once it was removed I would then resubmit. Kinda like trying to start with a fresh slate.

The problem is that the site will be removed for 6 months, which I never new at the time, now I have another 3 months before the site is indexed again(hopefully).

The only reason I did this was there was another domain that hijacked the site with the old redirect trick so I thought this would put an end to that as google was taking forever to fix the hijacking problems.

I found out at New Orleans that using robots text to get a site removed does so for 6 month's so it should be used with caution.

10:42 pm on July 18, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 20, 2002
posts:889
votes: 0


Kelowna

I assume you changed robots.txt and then used the Google remove URL. I'm not going to remove the URL through that service, so I assume I'll be able to unblock myself at a later date

9:47 pm on July 19, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Jan 3, 2004
posts:315
votes: 34


yes you are right, I did use the removal tool. I just thought people should know that the sites get removed for a 6 month period.
8:34 pm on July 23, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 20, 2002
posts:889
votes: 0


I've just checked my stats for yesterday and today and there's been a huge leap in traffic to my main site (not the one I've banned Googlebot from) - as in 400% increase in page impressions and nearly all originationg from a Google search referral. Yahoo was giving me nearly twice as much traffic as Google but now Google is giving me 4 times the traffic that Yahoo is.

Has there been a major Google update? Not judging by the threads here on WW.

I've just checked a number of searches I always use to judge my sites progress - and I am back in top position for each "widget words" and on the front page for "widget" - a position I last held in August last year. Earlier this week I didn't even appear for "widget words".

What conclusions can I reach at this early stage? Well, the experiment has been successful - something about one of my sites was dragging another down. I did have a time when I linked to my main site on every page (actually liked to the 4 main index pages on my main site so it was a bit excessive) There are pages on both sites that present the same information in different ways so there may be a duplicate content penalty of some sort. It certainly seems to me that this last 10 months of Google 'filtering' has been based on links (and Google IS a search engine ranking system based on links)

Loads of questions still, the site I banned Googlebot from is now up to #4 for its own name. Another site i have hasn't returned to its former position.

Personally I'm having a drink tonight - Czech lager.

9:27 pm on July 23, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 26, 2004
posts:109
votes: 0


Iguana, That's good news!
I had a similar situation last year. I bought a new URL to replace an old one. I temporarily had similar content on both sites and inter-linking. They both started to rank really well and then the new one slipped away to nothing. I banned Googlebot on the old one with a noindex, nofollow and the new one came back in ther SERPs within a few days. I experimented doing this a few times to make sure it wasn't an anomaly and always got the same results. I eventually left the no index, no follow on the old one for a few months and re-deployed it with completely different content (realizing the value of an old site because of sandboxing of new sites). I later took off the ban and they have both continued to rank very well for their respective content.
8:24 pm on July 24, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 20, 2002
posts:889
votes: 0


Thanks CathyM - you've also given me hope that I can allow Googlebot back to the site I banned it from in a month or two and maybe it will get decent ranking as well
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members