lucy24 - 3:37 am on Jun 28, 2013 (gmt 0)
Closely related question:
If a page comes with
(as some of my non-html pages do)
will this directive be honored in html pages that don't have a meta robots tag?
:: detour here to make sure the header has been working as intended with my non-page files ::
Here, again, the search engine will only see the header if it is allowed to receive the page. But it's an alternative way of conveying the same information. Useful if for example you don't want the index to hint at the existence of anything within a particular directory, even if you happened to forget the meta on one page.