homepage Welcome to WebmasterWorld Guest from 54.227.141.230
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Website
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

    
Permanent Self Removal
How to remove ones own site permanently from real visibility
LeaveMeAlone




msg:724073
 3:18 pm on Mar 21, 2006 (gmt 0)

I'll have to describe this one carefully:

OUR REQUIREMENT
We want to remove one of our sites from prime visibility. In other words we basically want to zap it as far as search engine ranking is concerned. It can stay in the index if necessary, but we want to zap the traffic.

I should clearly state here that this is OUR site. There is no funny business involved at all. We have good reason to get rid of this old site, and given that just putting a one page redirect there hasn't got rid, we need other options.

SOLUTIONS?
1) I know about the Google removal tool, but we don't want to see it suddenly re-appear in 180 days. Nor do we want to trust our own competence in terms of managing robots.txt when we change servers. We need another solution.

2) I thought about deliberately invoking a penalty. The problem is, which/how?

The obvious one is font color on same color background, but I think Google forgives you when this is corrected? It's not something I have done before.

Some people have suggested that we put certain inapproriate material on there, but we are simply not prepared to stoop to anything like that.

Another option may be to play around with 301/302s to sub-domains? That does seem messy though, and again, we would probably forget about it at some point.

Any thoughts or ideas on this one?

 

netmeg




msg:724074
 4:15 pm on Mar 21, 2006 (gmt 0)

Why don't you just replace whatever pages are there with blank pages, and THEN do the removal thing?

trillianjedi




msg:724075
 4:22 pm on Mar 21, 2006 (gmt 0)

You can add "nofollow/noindex" tags to pages.

I'm not quite sure why you don't trust yourself to manage robots.txt. If you want to block all engines, it's only two lines.

At the end of the day, whatever you do will require a certain element of management.

If the pages are for selected eyes only, you could always password protect the whole site.

TJ

kaled




msg:724076
 5:11 pm on Mar 21, 2006 (gmt 0)

If the site is entirely dynamic, you could add a robots noindex meta tag to every page.

However, robots.txt is undoubtedly the best solution.

Kaled.

netmeg




msg:724077
 6:28 pm on Mar 21, 2006 (gmt 0)

Except that there *are* spiders that don't follow it, whether by accident or design.

orbiter




msg:724078
 8:51 pm on Mar 21, 2006 (gmt 0)

I just went through this with one of my sites. For a number of reasons, I wanted it completed out of both Google's and Yahoo's indexes. Here's what I did...

I put up a robots.txt file with this...

User-agent: *
Disallow: /

I then used Google's removal tool. (I don't think Yahoo has one.) I submitted the request Friday evening. The site was removed from sometime yesterday afternoon.

Back to Yahoo: I resubmitted the URL to the site's home page to them. My hope is that they will check the robots.txt file if and when Slurp ever comes through.

And one last thing that I did: My site is entirely dynamic (I use ColdFusion). I've got code in the application.cfm file (a file that gets called before every request is processed) that looks at the user agent (browser type) and if it looks like a robot, I throw a 404 error and abort the request. By doing this, the spiders think the page is gone, but real users continue to see the page without any disruptions.

I hope this helps. Good luck!

g1smd




msg:724079
 12:38 am on Mar 22, 2006 (gmt 0)

Google will add the site back into their listings after 90 or 180 days. They will do that even if the site does not exist, or domain has expired.

If you make massive changes to the content, or just serve blank pages, Google will continue to show your old pages as Supplemental results for the next 2 or 3 years.

There is NO permanent removal method.

LeaveMeAlone




msg:724080
 11:39 am on Mar 22, 2006 (gmt 0)

"There is NO permanent removal method."

Maybe I didn't explain it well enough. The issue is prime visibility. Basically not ranking high on anything major. The site just sitting in a corner, albeit still in the index.

Remember the PR0 penalty? That is permanent(ish): site still there, but just not appearing on major searches.

We want something like that. Totally dampen down the traffic and visibility for good, without having to manage the robots.txt or other stuff in the future.

As I stated above, there are probably ways of attracting some sort of penalty like that, but we really don't want to do anything unethical or problematic, even though it is our site.

We just want to 'self-ban' or 'self-penalize' our site using the easiest and cleanest method.

Any ideas on this?

trillianjedi




msg:724081
 11:47 am on Mar 22, 2006 (gmt 0)

Totally dampen down the traffic and visibility for good, without having to manage the robots.txt or other stuff in the future.

Then you'll have to use "noindex/nofollow" as mentioned above, but to be honest robots.txt is the better solution.

We just want to 'self-ban' or 'self-penalize' our site using the easiest and cleanest method.

There's no guarantee that it will continue to work, and there's probably far more work involved in that (especially doing it in a way that doesn't impact the use of the site) than doing noindex/nofollow.

I would revisit your strategy here and do it the easy way, personally. And that's with robots.txt.

TJ

pageoneresults




msg:724082
 12:01 pm on Mar 22, 2006 (gmt 0)

I would revisit your strategy here and do it the easy way, personally. And that's with robots.txt.

I'll agree, the robots.txt is going to be part of the solution for those that obey it. As long as that robots.txt file remains in place and doesn't change the site is not going to reappear in 180 days, not according to Google guidelines.

Have you thought about putting everything behind a password protected area? That will surely stop it from being indexed. And, it is quick and easy and will effectively remove the site from all indexes. :)

LeaveMeAlone




msg:724083
 2:06 pm on Mar 22, 2006 (gmt 0)

We have thought long and hard about this, and robots.txt is not the route we want, for several reasons, some of which are internal to us. Hence, the question regarding 'self penalty'.

Text on same color background will attract a ban, but is it temporary (until repaired or the page changes)? I think so.

I guess I am looking for something like this but which will attract a penalty on a longer term basis. I do exclude putting offensive material on the site which I suspect would certainly work.

So it boils down to: what is the easiest way to get your own site into serious hot water, regarding a penalty?

pageoneresults




msg:724084
 2:22 pm on Mar 22, 2006 (gmt 0)

So it boils down to: what is the easiest way to get your own site into serious hot water, regarding a penalty?

Why would you want to incur a penalty? What if you decide to undo that at some time in the future?

I'm starting to question the motives behind the questions. ;)

trillianjedi




msg:724085
 2:24 pm on Mar 22, 2006 (gmt 0)

Text on same color background will attract a ban, but is it temporary (until repaired or the page changes)? I think so.

Just don't "repair" it afterwards. That might be easier than remembering to copy the robots.txt file to a new server. At least it's something you have to remember not to do, rather than remember to do ;-)

TJ

kaled




msg:724086
 2:28 pm on Mar 22, 2006 (gmt 0)

With respect, the penalty idea is ludicrous.

Whatever your reasons for disliking the robots.txt or (robots meta tag) solution, unless you want to add password protection (bad for users) or use cloaking (perhaps too complex) then it is just about your only option.

Kaled.

Just Guessing




msg:724087
 3:02 pm on Mar 22, 2006 (gmt 0)

So the question is - What is the easiest way to do some serious long term damage to a site, that can't be repaired quickly?

hmmm...

netmeg




msg:724088
 3:53 pm on Mar 22, 2006 (gmt 0)

(I'm trying to think of a hypothetical situation where one would want to do this to one's own site, within the parameters as thus far given, and I'm drawing a complete blank)

lammert




msg:724089
 5:11 pm on Mar 22, 2006 (gmt 0)

Some have suggested robots.txt and others adding the robots "noindex,nofollow" meta tag in the header.

I have experimented a lot with deleting content from the search engine indexes, both with the URL removal tool [webmasterworld.com] and robot meta tags and I would advice against the two options mentioned above and use the following meta tag instead:

<meta name="robots" content="noindex,follow">

There is a huge difference with this line compared with the other two options, i.e. Googlebot keeps crawling your site. In this way it reads the robot tag every time it fetches the file and the content will be removed from the index quickly.

With the other two options Googlebot is blocked access to other parts of the site with as side-effect that those pages are not removed from the index, but go supplemental. Going supplemental is what you have to avoid in all circumstances because once a URL is labeled "supplemental" in Google it is almost impossible to remove it from the index permanently.

So if you want to remove content from the SE indexes: let the bots in and let them spider to the deepest places of your site, but tell them to not index the page content.

CainIV




msg:724090
 6:21 pm on Mar 22, 2006 (gmt 0)

I would agree with Lambert here. The Google tool does not work correctly. Most pages simply are removed from the visible index then show up about 4-6 months later.

Meta tags is the best way to go should you not want robots.txt

Incurring a ban on your site to get it de-indexed is like driving into an oncoming car because you are our of gas.

lammert




msg:724091
 6:50 pm on Mar 22, 2006 (gmt 0)

Incurring a ban on your site to get it de-indexed is like driving into an oncoming car because you are our of gas.

It is more like driving agains an oncoming car because your brakes aren't working. It is an effective way to stop, but damage is--just as with a ban--quite severe.

The OP did not give much information about the reason of why he wants his site removed from the SE indexes, but I can think of many legitimate ones.

Copyright issues for example or a situation where the content of a site has been sold with the right of the original author to keeps his site up and running, as long as it doesn't compete with the new owner in the SERPs. Or what to think of a site which has a very strong community and doesn't need new lurkers from the search engines? Recently someone started a thread that hotlinkers from popular sites like myspace were stealing loads of bandwidth. No doubt that most of those surfers come via the search engines and going low profile can be a solution to avoid those useless visitors. Also Brett tried it some months ago when WebmasterWorld couldn't cope with the load of bots.

LeaveMeAlone




msg:724092
 6:59 pm on Mar 22, 2006 (gmt 0)

Sheesh. I didn't expect this to wrankle with so many people.

The idea of a self-penalty is that we don't have to worry about robots.txt, or meta-tags, or anything else. The site is zapped and low volume and that's it: clean and easy for us, for good. If you can't understand why we prefer that, I can't understand why you can't understand!

And then there are the inevetable folks who see machiavellian intent somehow. It's OUR site for goodness sake.

The REASONS aren't what this thread was about though. The thread was about the MECHANICS of what we want to do. It seems that no-one wants to discuss that though. I guess some fell on stoney ground.

pageoneresults




msg:724093
 7:18 pm on Mar 22, 2006 (gmt 0)

The thread was about the MECHANICS of what we want to do. It seems that no-one wants to discuss that though. I guess some fell on stoney ground.

Unfortunately the MECHANICS of what you want to do is something we would probably not want to share with the general public. There are of course your numerous threads at WebmasterWorld where penalties and bans are discussed. You could easily implement any number of techniques to get your site banned. From participating in bad link exchanges to using hidden text, it is all there for the taking.

What you won't find though are the MECHANICS for what you want to do. You'll need to play the role of mechanic and assemble your toolbox based on the various topics surrounding penalties and bans. And guess what, there are literally tens of thousands of them.

For many of us, it is difficult to look at a website from that perspective. If the site is toast now, then just let it go. Don't worry about it. Sounds like it was permanently toasted. Must have been something fairly severe to end up with that status.

trinorthlighting




msg:724094
 8:46 pm on Mar 22, 2006 (gmt 0)

Buy your own server and have your users dial up or remote access your files. Password protect your directories.

Have you even tried contacting google/msn/yahoo to see if they can permanently remove you from your index?

lammert




msg:724095
 11:18 pm on Mar 22, 2006 (gmt 0)

LeaveMeAlone,

Having read both your post in december last year and yours now, I think I understand your situation. You speak the truth in that the site is still under your control now, but my strong impression that it won't be in the near future and you want it removed from the SERPs in such a way that it can't recover even if it turnes into other hands.

Unfortunately I have to disappoint you at this point. There were several posts here of people appearantly buying banned domains which they didn't know at forehand. When owner and content is totally new you can as the new owner request Google to lift the ban on that domain. So whatever your reasons are to permanently hurt the rankings of this site, after it is out of your control bans can be lifted.

LeaveMeAlone




msg:724096
 1:12 am on Mar 23, 2006 (gmt 0)

OK, I give up. This particular site by the way will never leave our control. We just want to park it as it is, and then forget about it, with no maintenace needed and no new traffic. The ban would be the ideal solution for it for us, but I guess no-one will volunteer an easy mechanism. We'll have to experiement ourselves. No worries.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved