Am I doing it right?

Prevent Google to access one of my directory using robots.txt

[bashyam]

Hi..

I want to prevent google accessing one of my directory.. i have included the following syntax in robots.txt file... pls let me know its right!

User-Agent: Googlebot
Disallow: /<directory name to be blocked>/

[DaveAtIFG]

Looks OK to me! :)

If you want to brush up a little, here's a handy tutorial [searchengineworld.com] and a validator [searchengineworld.com].

[carfac]

Will Google's Imagebot follow googlebot instructions, or do you need a seperate lne for that?

dave

[jdMorgan]

carfac,

Google's normal crawler UA is "GoogleBot"
The ua for imagebot is "GoogleBot-Image"

Therefore disallowing "GoogleBot" disallows both 'bots because of prefix-matching, while disallowing GoogleBot-Image only disallows the image spider.

Jim

[jdMorgan]

bashyam,

Note lowercase "a":

User[b]-ag[/b]ent: Googlebot
Disallow: /<directory name to be blocked>/ 

Jim

[carfac]

Thanks Jim!

I was unsure.... I did not know if you had to match the bot name exactly...

dave