replacing robots.txt with a CGI

[theperlyking]

I would like to redirect robots.txt to a cgi so that I can log activity better (I dont have access to server logs :( ) and potentially have the option to do fancier things.

Does anyone know if there will be any pitfalls to this? Is there anyone doing this?

Thanks.

[littleman]

It really depends on how you do it. Do you have mod_rewrite? I don't think you want to use a redirect header.

[theperlyking]

Yep I have mod_rewrite, it's what i'm planning to use.
I can't imagine the SE balking at it but I was just wondering if anyone knew better.

[littleman]

Then this would do the trick.
RewriteEngine on
RewriteBase /
RewriteRule ^robots\.txt$ robots.pl

I do something similar.

[theperlyking]

Good, I know how to do it I was just scared :)

Thanks

[littleman]

Well, I guess you would judging from your name ;).

If you are real paranoid, I'd take the time to make sure the script's header matches up. It might be wise to make the script a 'nph-' and craft the header info your self depending on what the scripts output is.

[theperlyking]

Well my name is just fun, I kludge rather than code :)

Thanks for the real world advice I'll have a go.