Why is Google indexing our robots.txt file and showing it in search re

[Knocks]

For some reason, Google is indexing the robots.txt file for some of our sites and showing it in search results. See screenshots below.

Our robots.txt file is not linked from anywhere on the site and contains just the following:

User-agent: *
Crawl-delay: 5
This only happens for some sites. Why is this happening and how do we stop it?

[i.stack.imgur.com...]

Screenshot 1: Google Search console

[i.stack.imgur.com...]

Screenshot 2: Google search results

[not2easy]

Google has changed the way they deal with robots.txt files since last July, see: [webmasterworld.com...]

They may not see your file as a robot.txt file (despite its filename) because they ignore craw-delay directives and there are no instructions specifically for their bots.

[lucy24]

Screenshot 2: Google search results

Yes, but what search did you perform in order to get those results to come up? If I search for "cats", search results will not include every robots.txt in the world that mentions a directory called /cats/.

99 times out of a hundred, GSC messages about “indexed but not crawled” and the like are non-issues. Yes, in some abstract theoretical sense this content is indexed--but so what? If the idea really bugs you, add a rule like

<FilesMatch "\.(js|txt|xml)$">
Header set X-Robots-Tag "noindex"
</FilesMatch>

or equivalent if you're not on Apache. (This is an actual rule that I have, though I wasn't especially thinking of robots.txt when I made it.)

The fact that G ignores the Crawl-Delay directive is not your problem and does not require any action on your part.

[tangor]

robots.txt is a nothingburger from the get go. I allow all and sundry to have it, even if they don't follow the "rules".

No USER you want searched for robots.txt. Period.