Noindex in this directory

How can I say this to googlebot?

[joker197cinque]

How can I say GoogleBot NOT to index one directory?

I know about robots.txt but I never used it before.

I don't want Gbot index for example:

www.mydomain.com/private
www.mydomain.com/logs

ect....

but at the same time be SURE that all others bots and dir are allowed.

Thx in advance.

[vincevincevince]

rewritemod for user agent of google sending to 404 page?

[le_gber]

do you want the other bots to index these directory?

If not you can use:

User-agent: *
Disallow: /private
Disallow: /logs

And don't forget to put the robots.txt file on the root of your site.

Hope this helped

Leo

[vincevincevince]

some spiders are a bit jumpy, that's why i recommend using the modrewrite method if you want to be sure that other spiders don't get scared off. (if they never see it, they can't be scared by it!)

[jdMorgan]

joker197cinque,

This will do what you want to do. Copy it exactly, including both blank lines.

User-agent: Googlebot
Disallow: /private/
Disallow: /logs/

User-agent: *
Disallow:

For more info, see [robotstxt.org...]

Jim

[joker197cinque]

Thx all for your help.

I want to do this because some pages of mine (i suppose) penalized me as "crosslinking", and I want say Gbot not to find them again.

Am I right?

[killroy]

What about the issue of safesearch? Wouldn't a "partial" robots.txt remove him form safesearch results?

SN