Keeping my site OUT of search engines

[jmcnally]

I'm setting up a weblog for the employees of my company and don't want it to be indexed by any search engines. How can I keep this private site private? Sorry if this is a basic question!

[AjiNIMC]

The key word is robots files, just create a robots.txt file and stop the SEs.

This code in the robots.txt will stop all SEs to stop indexing all the pages, you can always configure according to your need.

# applies to all robots
User-agent: *
# disallow indexing of all pages
Disallow: /

But What I do in such cases is

www.domain.com/onlyforemployees.html and there is no link from any page to this www.domain.com/onlyforemployees.html page, so SEs cannt visit it.

Aji

[AjiNIMC]

and welcome to webmasterworld jmcnally.

Aji

[pageoneresults]

Hello jmcnally, Welcome to WebmasterWorld!

The robots.txt file is one way to prevent indexing, unfortunately it is not foolproof. The best way would be to make the blog username/password protected if you have that option.

If that option is not available, you might want to double up on any robots directives. You can utilize the robots.txt file and also include the META Robots Tag for safe measure...

<meta name="robots" content="none">

[Mohamed_E]

Depends on what you mean by "private".

If you just do not want it to be widely known you can ask search engines not to index it using the robots.txt file, see [robotstxt.org...]

To exclude all robots use:

# go away
User-agent: *
Disallow: /

Note that all you are doing is requesting robots not to index that site.

You can further help by attempting to avoid any external links to the site, if it has a large number of users you find that it is much more easily said than done.

The above will simply minimize the chance that a casual browser will find your site. The only way to guarantee secrecy is by using either password protection or a firewall.

[victor]

Or you could think about serverside checks that all incoming requests are from specific IP addresses -- read up on .htaccess in this forum. A sort of roll-your-own firewall.

[jmcnally]

Thanks everyone. I've implemented both suggestions (<meta> tags and robots.txt file).

Is there any way to block my site from showing up in referrer logs of sites I link to? That's another way that we might get external unwanted visitors...

Thanks again!

James