Forum Moderators: open
On my site, i have some confidential directory (such passwaord directory).
I really don t want them to be crawl by a bot.
First, i use a .htaccess to avoid anyone to come.
Second, there is no link anywhere to go to this URL (so noone can know their existance)
I want to know: Isn't it dangerous to put this kind of directory in the robots.txt?
Thx for all answers :)
There are no stupid questions around here, so please, feel free to ask all the questions you want.
You are correct that it is somewhat dangerous to list specifics in your robots.txt file. Any qualified hacker could probably gain access to that directory. I'm not familiar with .htaccess but have read enough to know that you are doing the right thing.
If the directory is password protected, I don't think the spiders will get in there anyway, no need to place it in your robots.txt file.
