Forum Moderators: phranque

Message Too Old, No Replies

mod_rewrite to remove variable

Is it possible to remove the variable from the URL?

         

mangotude

9:06 pm on Mar 27, 2005 (gmt 0)

10+ Year Member



Hello,

I've posted a while ago (was 18 months ago ...that went quick) about a quick mod_rewrite - it was fairly straightforward, taking a dynamic page and creating a static


RewriteEngine on
RewriteRule ^profile_(.*)\.html$ /index.php?page=view_profile&id=$1 [L]

for a different site I have a similar issue - the original URL would be

[yyyyy.com...]

This page is sent to my payment processor. However, I have signed up for additional processor, and they will not accept this (or any) variation from the predefined page.

So what I am asking is it possible to take the URL above and end up with something like

[yyyyy.com...]

or

[yyyyy.com...]

I have looked for a similar request without any joy, and my gut feeling is 'probably not', but since I'm usually wrong on these things I thought I should ask here.

Thanks in advance for any help, and Happy Easter to all concerned.

James

HughMungus

9:15 pm on Mar 27, 2005 (gmt 0)

mangotude

9:34 pm on Mar 27, 2005 (gmt 0)

10+ Year Member



Hi,

Thanks for this...I've looked through it and I'm not sure what part is relevant (it's a 41 message post). If you could point it out for me, that would be appreciated.

It's not a question of laziness, it's that I don't understand the code.

Many thanks

James

HughMungus

9:54 pm on Mar 27, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



message #4

mangotude

10:04 pm on Mar 27, 2005 (gmt 0)

10+ Year Member



Hi

Thanks for clarifying. The mod_rewrite in that message is doing the opposite of what I want - taking a fixed (and fake) URL and then changing it back to the query string - is it then possible to 'invert' the script?

Or are you referring to the PHP script itself - I ask because my script is perl, and I have no experience of PHP.

THanks again

mangotude

10:58 am on Mar 28, 2005 (gmt 0)

10+ Year Member



I thought I should rephrase my request for clarity.

Is it possible to use mod_rewrite to remove the variable from the URL (Yes/No)

If it's yes, then i'll go off and try to work out the code myself. If its no, then i'll work out something on the form display or cgi.

Many thanks

sitz

4:19 am on Mar 29, 2005 (gmt 0)

10+ Year Member



Yes; if you want to remove the query string completely, rewrite to a URL with a trailing '?', and nothing after it. If you want to remove a particular segment of a query string, you can use RewriteCond to slice out the piece(s) you don't want, and Rewrite to the remainder:

Given:

GET /foo/bar.html?baz=bing&quux=doink&frisbee=frotz

Remove quux=doink:


RewriteCond %{QUERY_STRING} ^(.*&)?quux=doink(&.*)?
RewriteRule ^(/foo/bar.html$) $1?%1%2 [L]

[edited by: jdMorgan at 6:53 am (utc) on Mar. 29, 2005]
[edit reason] Fixed unintentional smiley-face in code. [/edit]

mangotude

9:35 pm on Mar 29, 2005 (gmt 0)

10+ Year Member



Hi,

Thanks for this - its sort of along the lines of what I was hoping was possible. However it's a case of "it's great...but" - the great is that

Yes; if you want to remove the query string completely, rewrite to a URL with a trailing '?', and nothing after it.

Is exactly what I'm hoping to achieve

and the but is that the variable is always...well..variable, in that it would be different each time - as its an order number.

so it would be

[yyyyy.com...]

113193
113194
etc

so while it is variable, it is also predictable - so would it be possible to [I apologise for the rough terms here, i'm inventing as I don't know the correct usages] 'mass block' these - so if a URL appears as above, and the number is within say 113192-211398, then the number will be stripped out.

So I guess the questions are

1> Is this possible?
2> If possible, would a separate rewrite line be required for each number?
3> if separate rewrite lines are required, would that place the server under an intolerable burden - there wouldn't be more than 100 requests or orders a day.
4> Am i making it far too complicated, and can I simply 'hack off' the URL once it gets to '?'

Many thanks for your ongoing help.

James

jdMorgan

10:22 pm on Mar 29, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



This is a very simple problem. You just rewrite any URL with that variable name (show) to the same URL as requested, and stick a "?" on the end of the new URL, and poof, the query string is gone.

RewriteCond %{QUERY_STRING} ^show=
RewriteRule ^orderread\.pl$ http://www.example.com/orderread.pl? [R=301,L]

Poof. Blank. Gone.

Jim

mangotude

11:07 pm on Mar 31, 2005 (gmt 0)

10+ Year Member



Hi Jim,

I had hoped to post once I had this running correctly, but I've yet to do so. So I just wanted to say it's appreciated, and I'll let you know once I've got it working ok.

I've got mod_rewrite working fine, and I did an initial 301 check with a code you suggested elsewhere


Options +FollowSymLinks
RewriteEngine on
RewriteRule ^no\_such\_file\.html$ /index.html [R=301,L]

So if mod_rewrite is working, and 301 is possible, then I can only think that it's there's an issue with cgi-bin, as I find it highly unlikely your code was off.

Hopefully I'll have a positive update soon.

Best regards,

James

EDIT - I've looked through the logs and I think the main code Im getting is 304 (which I believe I should get anyway if i'm calling up a file...?show=123456), but occassionally i've had 500 630. I've seen 500 before, but im unsure about 630

jdMorgan

4:43 am on Apr 1, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



> there's an issue with cgi-bin.

It is common to have cgi-bin "aliased" on servers. This often means that control is taken away from you before your code can run. The simplest way to find out is to try to rewrite *anything* in the cgi-bin with a very simple rewriterule. If it doesn't work, then it's likely that your code is never executed. Plan B is to ask the server admin to fix this problem, and Plan C is to live with it... :(

304 - Not Modified. The file has not been modified since it was last cached by the client.
500 - Server Error. Webmaster or server admin has some serious debugging to do.
630 - Time for Beer. Quitting time for those who work past 5:00. (I've never heard of this code before.)

Jim

mangotude

1:19 pm on Apr 1, 2005 (gmt 0)

10+ Year Member



Hi Jim,

Substitute beer for Guinness and you were bang on :)

I tried a couple of simple redirects like you suggested in cgi-bin and they don't seem to be doing anything. So at this point I assume it's an aliasing problem.

What should I ask my host to do - to remove the aliasing? - is that simple? - and then is there any real security concerns from the action he needs to take?

Best regards,

James

jdMorgan

3:58 pm on Apr 1, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



James,

The reason I stuck a :( in my post is that it's very unlikely that they'll change anything for you.

Another option I should have mentioned is to create your own cgi-bin directory (call it something like usr-bin or something) and then put only those scripts which need their calling parms rewritten into that new directory. Then, yes, you will have to be extremely careful about filesystem write and execute permissions in that directory, and wall it off from the HTTP side with .htaccess as soldily as possible while maintaining functionality.

You could also change all references to the cgi-bin directory to point to your new directory, and then have "mini-scripts" in the new directory which invoke (include) the real cgi-bin scripts only after the rewrites are done.

There are several ways to work-around the problem, but none simple and all with security concerns that need to be addressed. And there are likely to be side-effects as well, due the the protections on cgi-bin. All I can recommend is to experiment with the methods in the simplest way possible, and then expand on any method that seems to work in your server environment.

Jim

mangotude

11:11 pm on Apr 1, 2005 (gmt 0)

10+ Year Member



Hi Jim,

My hosts have been excellent so far, and I have a dedicated server which was set up by them on my behalf - so unless there's a shocking security issue with aliases I'll stick with unfounded optimism and see what they say.

Failing that, I'll consider the options you've mentioned very carefully, as I feel I could do more harm than good.

Best regards,

James

mangotude

5:01 pm on Apr 2, 2005 (gmt 0)

10+ Year Member



Ok,

Now I'm bemused.

I've checked with my host - apparently no aliases.

I've got my error log and there's no errors when i try to run the page that would be rewritten.

So I took the basic 301 i mentioned in an earlier post and used .pl instead of .html

And there I get
script not found or unable to stat: /var/www/myusername/cgi-bin/no_such_file.pl

So I'm lost now, because mod_rewrite works on basic html, apparently there are no aliases, but in cgi-bin it's simply not even trying to redirect it.

Best regards,

James

jdMorgan

6:20 pm on Apr 2, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I wonder if you've got a LoadModule list ordering problem. If you are on Apache 1.x, ask your host to make sure that mod_rewrite appears in the LoadModule list in httpd.conf *after* mod_cgi.

In Apache 1.x, modules are executed in reverse order of that shown in the LoadModule list. Therefore, if mod_rewrite appears in the list before mod_cgi, then any request for a cgi will not be processed through mod_rewrite.

We run into this here all the time with home-brewed php installations, because people naturally assume that they should politely add php to the end of the list; When they do, mod_rewrite and mod_auth, and many other modules mysteriously won't work for php requests, because php is processed first.

This does not apply to Apache 2.x which uses a different method, allowing modules to declare their own priority -- and I still haven't done much research into it.

Jim

claus

6:24 pm on Apr 2, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>> /var/www/myusername/cgi-bin/no_such_file.pl

Is your web root in: "/var/www/myusername/www" or similar?

If your cgi-bin is outside your web root, and your ".htaccess" file is in your web root, then your ".htaccess" has no influence over your cgi-bin. The files and folders you want to change must be in the same folder as your ".htaccess" or in subfolders of the folder holding the ".htaccess" file.

Anyway, on the thread topic:

Here's How to catch a query string with Apache [webmasterworld.com] (although that problem might be solved by now)

mangotude

7:49 pm on Apr 2, 2005 (gmt 0)

10+ Year Member



Hi,

The server is running Apache 2....cgi-bin is outside my web root, but .htaccess was put directly in the cgi-bin folder.

This is probably completely wrong, and i'm grasping at straws...but could it be an issue of something like "ExecCGI" or something along those lines?

Best regards,

James

jdMorgan

8:40 pm on Apr 2, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



If cgi-bin is outside your web root, then by definition it is inaccessible via HTTP. So .htaccess won't have any effect, because the mechanism used to invoke the scripts is entirely server-side and doesn't invoke .HTaccess (the HT in .htaccess stands for HTTP).

So, a basic question comes up: How is orderread.pl invoked? Is it included in an HTML or php page?

We may just have gone around in circels for a few days based on false assumptions and loose terminology...

Jim

mangotude

8:57 pm on Apr 2, 2005 (gmt 0)

10+ Year Member



Hi Jim,

If I have caused a few days of wasted time, my apologies.

Just to make absolutely clear:

on accessing FTP I have the following:

var/www/myusername

cgi-bin
homes
logs
public_html

The basic set up is as follows:

I go to form.html
complete form -
*which is sent to writeorder.pl
*writeorder.pl writes to flat txt database,
*sends email of order to me,
*and then calls up orderread.pl?show=ID
*orderread.pl uses 'template.html'

the addresses would be:
www.yyyyy.com/form.html

www.yyyyy.com/cgi-bin/orderread.pl?show=ID

(i can also through the browser visit
www.yyyyy.com/cgi-bin/orderead.pl - showing all order details
or any specific order number - which is why I thought it came under http control)

I hope that's all the required information.

Best regards,

James

claus

9:11 pm on Apr 2, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Uhm.. what i do is to put a central .htaccess file outside of both my web root and my cgi bin - ie. in my document root (not my web root). That one controls the base rules for both the cgi-bin and the web root, plus whatever other stuff i might have outside the web root.

In your case, the base ".htaccess" file would be in the folder "var/www/myusername/"

It works for me, but perhaps my server settings are special, i don't know.

Now, here's a wild guess. It's a total shot in the dark i found on some web page and i'm not sure if it will work, but it will not harm trying it:

CHMOD your .htaccess in the cgi-bin to 644 (you don't want your .htaccess file to be executable, as it isn't a script, but it should be world readable). (this part edited for clarity)

If .htaccess really does not work in your cgi-bin folder, it will be because the directive

AllowOverride AuthConfig
is not set for your cgi-bin folder. This is done in the "httpd.conf" file, which you probably can't modify if you are not running your own server. In this case, your host will be able to modify it for you.

mangotude

10:31 pm on Apr 2, 2005 (gmt 0)

10+ Year Member



So...should I have just one .htaccess in the document root, which then defines all the other folders? If so, can I just take the existing code and rework it for a specific folder, or do I need to create a 'directive' for the directory?

I'll check up on the

AllowOverride AuthConfig
as well

claus

12:02 am on Apr 3, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I believe that you could just rework it, adjusting the paths as necessary.

By the way, i was wrong above - my "document root" is actually also a "web root" (although i don't use it as such), meaning that content can be served from it. I recall that when i started on that machine, my root-level files on one domain were actually being displayed from that folder.

I have now reworked my folder structure, so that this location does not hold any content that is "public" at all. That is, my folder structure (on the server) is not the same as my URL structure (on the www). So, the cgi-bin that is now outside my web root actually used to be a subfolder of my web root (which is why i can manipulate it from the root .htaccess file).

I'm not sure if it's important for your case, but Jim's post above made me remember it.

So, if your cgi-bin is really outside your web root, then it is connected to the rest of your site via a setting in the httpd.conf file, meaning that your web root will not get read before the cgi-bin, so your root htaccess file has no effect (as Jim has already said).

Anyway, let's see if you have "AuthConfig" enabled or not. If you have you should be able to place a .htaccess file inside the cgi-bin directory.

---

If all else fails, you could create another folder inside your web root and use that one as your new cgi-bin. For this to happen, the folder needs to be CHMOD 755 (i think it's 755 anyway, but you can check with your present cgi-bin) and this should be inside the .htaccess of that folder:

--------------------------- 
Options +ExecCGI
AddType application/x-httpd-cgi .cgi
AddType application/x-httpd-cgi .pl
AddHandler cgi-script .cgi .pl
---------------------------

mangotude

9:06 pm on Apr 3, 2005 (gmt 0)

10+ Year Member



Hi.

I got AuthConfig running and then i got a 500 error, which was "RewriteEngine not allowed here" so I added to cgi-bin "AllowOverride FileInfo", and it now works fine.

Thank you Claus and Jim. Much Appreciated.

Best regards,

James