The canonical URL for a folder and for the index page in that folder is one that ends with a trailing slash and does not mention the actual name of the index page document if one exists.
You should redirect all index.* and any other such forms to the canonical URL. This is a simple job using a RewriteCond to detect that the request came from the web and a RewriteRule to issue the 301 redirect.
There's several hundred threads with basic code that can easily be adapted to handle multiple alternative extensions in this very forum. It's a question that's asked several times each month.