Receptional_Andy - 10:59 am on Aug 20, 2008 (gmt 0)

If you're on Apache and can use htaccess directives and mod_rewrite, you'll find a fantastic guide to avoiding all types of duplicate content in the thread A guide to fixing duplicate content & URL issues on Apache [webmasterworld.com].

That thread is pretty comprehensive and may seem a bit intimidating if you aren't used to working with mod_rewrite, but is well worth taking the time to work through and understand.

I wouldn't recommend excluding index documents via robots exclusion since there's the potential for unwanted side effects.

If you haven't already, you should also get into the habit of making sure your site's internal links point to a directory without a document index - i.e. link to www.example.com/ not www.example.com/index.htm and www.example.com/directory/ not www.example.com/directory/index.htm