Welcome to WebmasterWorld Guest from 54.242.138.24

Forum Moderators: goodroi

Message Too Old, No Replies

replacing robots.txt with a CGI

     

theperlyking

9:21 am on Apr 25, 2001 (gmt 0)

10+ Year Member



I would like to redirect robots.txt to a cgi so that I can log activity better (I dont have access to server logs :( ) and potentially have the option to do fancier things.

Does anyone know if there will be any pitfalls to this? Is there anyone doing this?

Thanks.

littleman

4:58 pm on Apr 25, 2001 (gmt 0)

WebmasterWorld Senior Member littleman is a WebmasterWorld Top Contributor of All Time 10+ Year Member



It really depends on how you do it. Do you have mod_rewrite? I don't think you want to use a redirect header.

theperlyking

5:18 pm on Apr 25, 2001 (gmt 0)

10+ Year Member



Yep I have mod_rewrite, it's what i'm planning to use.
I can't imagine the SE balking at it but I was just wondering if anyone knew better.

littleman

5:22 pm on Apr 25, 2001 (gmt 0)

WebmasterWorld Senior Member littleman is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Then this would do the trick.
RewriteEngine on
RewriteBase /
RewriteRule ^robots\.txt$ robots.pl

I do something similar.

theperlyking

6:05 pm on Apr 25, 2001 (gmt 0)

10+ Year Member



Good, I know how to do it I was just scared :)

Thanks

littleman

6:17 pm on Apr 25, 2001 (gmt 0)

WebmasterWorld Senior Member littleman is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Well, I guess you would judging from your name ;).

If you are real paranoid, I'd take the time to make sure the script's header matches up. It might be wise to make the script a 'nph-' and craft the header info your self depending on what the scripts output is.

theperlyking

7:42 pm on Apr 25, 2001 (gmt 0)

10+ Year Member



Well my name is just fun, I kludge rather than code :)

Thanks for the real world advice I'll have a go.

 

Featured Threads

Hot Threads This Week

Hot Threads This Month