using [N] to recursively build a querystring

[TheAlchemist]

hello all,

I'm going nuts trying to do make the following rules work:

RewriteRule .*/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~
([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/test-([0-9]+)$
test.php?CatID=$21&$1=$2&$3=$4&$5=$6&$7=$8&$9=$10&$11=$12&$13=$14&$15=$16&$17=$18&$19=$20 [L]
RewriteRule .*/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~
([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/test-([0-9]+)$
test.php?CatID=$19&$1=$2&$3=$4&$5=$6&$7=$8&$9=$10&$11=$12&$13=$14&$15=$16&$17=$18 [L]
RewriteRule .*/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~
([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/test-([0-9]+)$ test.php?CatID=$17&$1=$2&$3=$4&$5=$6&$7=$8&$9=$10&$11=$12&$13=$14&$15=$16 [L]
RewriteRule .*/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~
([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/test-([0-9]+)$
test.php?CatID=$15&$1=$2&$3=$4&$5=$6&$7=$8&$9=$10&$11=$12&$13=$14 [L]
RewriteRule .*/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/test-([0-9]+)$
test.php?CatID=$13&$1=$2&$3=$4&$5=$6&$7=$8&$9=$10&$11=$12 [L]
RewriteRule .*/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/test-([0-9]+)$
test.php?CatID=$11&$1=$2&$3=$4&$5=$6&$7=$8&$9=$10 [L]
RewriteRule .*/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/test-([0-9]+)$ test.php?CatID=$9&$1=$2&$3=$4&$5=$6&$7=$8 [L]
RewriteRule .*/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/test-([0-9]+)$ test.php?CatID=$7&$1=$2&$3=$4&$5=$6 [L]
RewriteRule .*/([a-zA-Z0-9_]+)~([^/]*)/([a-zA-Z0-9_]+)~([^/]*)/test-([0-9]+)$ test.php?CatID=$5&$1=$2&$3=$4 [L]
RewriteRule .*/([a-zA-Z0-9_]+)~([^/]*)/test-([0-9]+)$ test.php?CatID=$3&$1=$2 [L]

the last 4 rules DO work. all the other ones don't. that is because apache variables only go from $1 to $9. I need more than that and I don't know how many key/value pairs will be there. So now I'm trying to find a way to accomplish the same task with just one line. I know it has to do with the [N] but when I run test code it looks like it gets into an infinite loop. please take a look at my rules and see what I'm trying to accomplish. basically I need to replace all ~ with = and build the querystring from there.

thank you very much in advance

[edited by: jdMorgan at 6:48 pm (utc) on July 4, 2007]
[edit reason] Alleviated horizontal scrolling. [/edit]

[jdMorgan]

OK, the dizzyness has worn off a bit... :)

Take a look at this code. It may not be quite what you want, and due to typos and possible logic errors, it probably won't work. But it demonstrates a technique to do what you're trying to do:

# Add a tag to the URL to make it easy to identify as a work-in-progress
RewriteRule ^(([a-z0-9_]+~[^/]+/)+test-([0-9]+))$ temp1>$1 [NC]
#
# If only one name/value pair, move it to query string, change URL-path to test.php, and quit
RewriteRule ^temp1>([a-z0-9_]+)~([^/]+)/test-([0-9]+)$ test.php?CatID=$3&$1=$2 [NC,L]
#
# If two or more sets of name/value pairs, move first pair to query string and declare CatID
RewriteRule ^temp1>([a-z0-9_]+)~([^/]+)/(([a-z0-9_]~[^/]+/)+)test-([0-9]+)$ temp2>$3?CatID=$5&$1=$2 [NC]
#
# Move remaining name/value pairs (except the last) to query string using mod_rewrite [N] restart and [QSA]
RewriteRule ^temp2>([a-z0-9_]+)~([^/]+)/(([a-z0-9_]~[^/]+/)+)$ temp2>$3?$1=$2 [NC,QSA,N]
#
# Move last name/value pair to query string, change URL-path to test.php, and quit
RewriteRule ^temp2>([a-z0-9_]+)~([^/]+)$ test.php?$1=$2 [NC,QSA,L]

The "temp" URL-prefix tags make pattern-matching more efficient, and make it easier understand and control the code execution and determine when we're done and need to change the URL-path to its final value of test.php.

I used the [NC] flag to make pattern-matching case-insensitive. This allows us to use [a-z0-9_] instead of [A-Za-z0-9_] and therefore eliminates one of the otherwise-required character-range checks. I'd expect more than 25% speed-up from this change alone.

When using nested parentheses as above, count left-parentheses to determine the back-reference number.

Apache has a nasty mod_rewrite bug [archive.apache.org] that may crop up when doing multiple rewrites like this. If the bug afflicts this code, there *is* a way to fix it, but it makes the code even more dauntingly complex and obscure. The symptom of the bus is "unexpected extra copies" of the URL-path appearing in the rewritten URL. This bug was supposed to have been be fixed in Apache 2.x, but my tests have determined that it is still present.

If you have the option to modify your test.php script, I'd suggest doing so; Simply rewrite *any* URL that has any number of the required name/value pairs to the script, then let the script retrieve the name/value pairs directly from the REQUEST_URI. This would be *much* more efficient than using the [N] flag in mod_rewrite, which will cause *all* rules preceding this section of code to be re-executed for *every* name/value pair except the first and last. This is slow, and depending on your pre-existing rules, could have undesired side-effects. As the above code demonstrates, flow-control in mod_rewrite is very difficult, because it is not a structured language. Neither does it have the ability to "jump" to line numbers or named labels.

Jim

[TheAlchemist]

ok. first of all, thank you very much for your efforts. I'm not quite sure I understand all of it. but I'm trying. I tested your code. it works if there is only one key/value pair. otherwise, I get this error:

Not Found
The requested URL /temp1>Country~USA/State~NY/test-1 was not found on this server.

Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.

temp1 is not supposed to be there, is it?:)

thank you again

[jdMorgan]

OK, yeah, I see a problem. But I didn't test the first version, and I didn't test this one. I intend this only as a demonstration of a method to recursively move URL path-parts to the query string. Due to practical considerations, I'm afraid it's up to you to modify it to suit your needs, test it, and get it working. This isn't exactly a project that anyone else here can set up and test...

Because test~<catid> can be (and is) dropped from the URL, the different "testN>" prefix strings may not be needed to differentiate the first multi-parameter rule that grabs the catid from the ones that follow.

# Add a tag to the URL to make it easy to identify as a work-in-progress
RewriteRule ^(([a-z0-9_]+~[^/]+/)+test-([0-9]+))$ temp>$1 [NC]
#
# If only one name/value pair, move it to query string, change URL-path to test.php, and quit
RewriteRule ^temp>([a-z0-9_]+)~([^/]+)/test-([0-9]+)$ test.php?CatID=$3&$1=$2 [NC,L]
#
# If two or more sets of name/value pairs, move first pair to query string declare CatID, & drop "test~<catid>"
RewriteRule ^temp>([a-z0-9_]+)~([^/]+)/(([a-z0-9_]~[^/]+/)+)test-([0-9]+)$ temp>$3?CatID=$5&$1=$2 [NC]
#
# Move remaining name/value pairs (except the last) to query string using mod_rewrite [N] restart and [QSA]
RewriteRule ^temp>([a-z0-9_]+)~([^/]+)/(([a-z0-9_]~[^/]+/)+)$ temp>$3?$1=$2 [NC,QSA,N]
#
# Move last name/value pair to query string, change URL-path to test.php, and quit
RewriteRule ^temp>([a-z0-9_]+)~([^/]+)$ test.php?$1=$2 [NC,QSA,L]

Your second error message indicates that you have declared a custom 404 error page using the ErrorDocument 404 directive, but that that page does not exist. Either remove the ErrorDocument 404 directive, or create the 404 error page that it names.

Jim

[TheAlchemist]

hello,

thanks again for your help. however, this time it's not working at all, not even with 1 key/value pair.

thanks anyway.