how to set joomla robots.txt file

Joomla Robots

[cafedelmar]

Hi all
Could someone tell me what is the best setup for Joomla robots.txt file.
right now I use standard:

User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /components/
Disallow: /editor/
Disallow: /help/
Disallow: /includes/
Disallow: /language/
Disallow: /mambots/
Disallow: /media/
Disallow: /modules/
Disallow: /templates/
Disallow: /installation/

is this ok?
greets

[Reid]

I would remove the trailing slashes if you want to completely protect those folders. As long as you dont have any files with those names.
Example what would "GET /administrator" return? Disallow: /administrator/ would not stop it.

robots.txt is based on 'prefix match' beginning at the root "/"

Disallow: /admin

would disalow any URL with the prefix /admin
including:
/admin
/adminisrator
/administrator/
/adminibator
/admin/
/admin.html
/admin.php

It simply matches the prefix of the URL beginning at the root.
This will not prevent any robot from access it simply "asks them not to go there"
For real protection you may want to use server commands like .htaccess or equivalent.