phranque - 9:20 am on Sep 17, 2012 (gmt 0)

the index for a directory should be served from the directory's url which ends with a trailing slash:
http://www.example.com/subdirectory/

the next step is that you should configure your server to specify the default directory index document - say index.html - and any requests for the default directory index document, for example http://www.example.com/subdirectory/index.html, should be externally redirected with a 301 status code to the trailing slash url:
http://www.example.com/subdirectory/

from your description it sounds like the content in the what-is-here.html file should actually be in the index.html file.

if http://www.example.com/subdirectory/what-is-here.html has been indexed or if you are getting any requests for that file, for example a returning visitor who has bookmarked that file, those requests should also get 301 redirected to the trailing slash url:
http://www.example.com/subdirectory/


regarding your meta noindexed content showing up in the index - that shouldn't happen unless you are excluding this content from crawling by disallowing in robots.txt.
in this case you would typically see this content in a search snippet with no description and in its place is text similar to this:

A description for this result is not available because of this site's robots.txt – learn more

in this case you should allow crawling so that the indexer can see the neta noindex element in the document.


the only way to noindex an image file or any other resource that is not an html document is to use the X-Robots-Tag HTTP Response header.


your site map script may help a search engine discover urls on your site but the absence of a url does nothing to exclude that resource from crawling nor from indexing as an incomplete snippet if the url is discovered elsewhere.