allow robots.txt access but block txt files reading

[grigione]

in my htaccess file i have folllowing


<Files ~ "\.(tpl|txt)$">
Order deny,allow
Deny from all
SetEnvIfNoCase User-Agent "Googlebot" goodbot
Allow from env=goodbot
</Files>

I want block all to access tpl and txt files but permit to google bot crawler to access to robots.txt

this configuration working in a server but in an other don't working

any alternative?

[g1smd]

Something like this to physically block access:

RewriteCond %{REQUEST_URI} !^/robots.txt$
RewriteRule \.t(xt|pl)$ - [F]

All agents should see robots.txt otherwise nothing is disallowed.

[grigione]

ok thank you very much code working But only a question is possible also hide visualization of robots.txt at human eyes?

[mememax]

I think the only way is to use the ip filter in htaccess but in this way you must be sure to insert all google's (and other user agents) IPs.

[lucy24]

Google may have already indexed your robots.txt-- this is common though not universal-- and in that case there's not much point to blocking humans. If you wanted to keep humans from knowing what's there, you could set a timer on robots.txt so the file only stays open for, say, half a second. Or a millisecond or whatever. Loads of time for a robot to assimilate it, but not enough for human eyeballs and brains.