Odd behafior of mod rewrite, help!

Is the bug in my brain or in the server?

[Dan_Sitar]

Hi masters of the .htaccess file,

My domain is on a shared host that is driven by Apache v2.2.3 (pleaze, tell them to upgrade,lol). I want to control WHO can hotlink my images, which I want to rewrite as always the same image. But I want those allowed to get a hotlinking.gif if they try to link to any of my medias and webfonts. And of coarse everybody else but me would get my hotlinking annimation for all images, medias and webfonts.

Being new at this, I had to study the docs of the current Apache. Everything that I have tried did not work as expected. It was confusing to see in my log that some directives was not recognized, even if the syntax was good. Like the <If>, <ElseIf>, <Else> directives that I used often in JavaScript, which were perfect for my case. No wonder, those are for Apache v2.4 while I found out later on that I am on the 2.2 version!

Thank God I have found this forum, in which I have learned not to mix “mod_setenvIf” and “mod_rewrite”! So I am back to the basic with only the mod_rewrite.

Can you explain to me the Odd behavior of this test?

I have created four images: ‘is_gif’, ‘is_jpg’, ‘is_png’ and ‘is_NOT_image’ to test the behafior of many “RewriteCond & RewriteRule” in the same .htaccess file.

DirectoryIndex /hotlink/index.html
RewriteEngine on
RewriteBase /hotlink/

RewriteCond %{REQUEST_URI} "[a-z0-9_-]+\.jpg$" [NC]
RewriteRule ([^/.]+)\.jpg$ is_jpg.gif [L]

RewriteCond %{REQUEST_URI} "[a-z0-9_-]+\.png$" [NC]
RewriteRule ([^/.]+)\.png$ is_png.gif [L]

# RewriteCond %{REQUEST_URI} "[a-z0-9_-]+\.gif$" [NC]
# RewriteRule ([^/.]+)\.gif$ is_gif.gif [L]

RewriteRule ([^/.]+)\..* is_NOT_image.gif [L]

This script works fine when I request these images that exist:

mydomain.com/hotlink/image.jpg , ,sends ‘is_jpg.gif’
mydomain.com/hotlink/image.png , sends ‘is_png.gif’
mydomain.com/hotlink/image.gif , , sends ‘is_NOT_image.gif’
mydomain.com/hotlink/what-ever , , sends ‘is_NOT_image.gif’

The ODD behafior:

It is when I remove the ‘#’ from the “is_gif” RewriteRule, (a copy & paste of one of those “RewriteCond & RewriteRule” where I changed only the extension,) It Monopolizes ALL The Results:

mydomain.com/hotlink/image.jpg , ,sends ‘is_gif.gif’
mydomain.com/hotlink/image.png , sends ‘is_gif.gif’
mydomain.com/hotlink/image.gif , , sends ‘is_gif.gif’
mydomain.com/hotlink/what-ever , , sends ‘is_gif.gif’

I give up, do you? Thanks.

[lucy24]

I have learned not to mix “mod_setenvIf” and “mod_rewrite”! So I am back to the basic with only the mod_rewrite

If using both mod_setenvif and mod_rewrite is bad, I am in trouble because I use both. Now, combining mod_alias with mod_rewrite... that's dangerous!

Do your four images all look entirely different, so you can see with your eyeballs which one is coming up even though the address bar doesn't change? Good thinking.

RewriteCond %{REQUEST_URI} "[a-z0-9_-]+\.jpg$" [NC]
RewriteRule ([^/.]+)\.jpg$ is_jpg.gif [L]

RewriteCond %{REQUEST_URI} "[a-z0-9_-]+\.png$" [NC]
RewriteRule ([^/.]+)\.png$ is_png.gif [L]

RewriteCond %{REQUEST_URI} "[a-z0-9_-]+\.gif$" [NC]
RewriteRule ([^/.]+)\.gif$ is_gif.gif [L]

Something further along the line is causing a Redirect instead of the intended Rewrite, so all your different gifs are looping back and showing up again in mod_rewrite-- and now they all feed into the third rule. So the problem is actually not in the rule itself; in fact the rule is making some other unrelated problem become visible.

You generally don't need a RewriteBase, especially not in htaccess. It's safer to say outright

RewriteRule \.gif$ /hotlink/is_gif.gif [L]

Note, unrelatedly, that you don't need to say anything about what comes before the extension, since you won't be capturing it. Matter of fact, for testing purposes you don't need the current Conditions at all, although you'll need conditions later when you fine-tune the rule.

RewriteRule \.(gif|png|jpe?g)$ /hotlink/is_$1.gif [L]

(I stress "for testing purposes" because this version has to be case-sensitive.)

[g1smd]

Use a preceding negaive match RewriteCond looking at REQUEST_URI to exclude already rewritten requests otherwise you can end up with an infinite loop for some requests.

Don't forget that your browser caches the images by requested URL (not by internal server filename) so you need to clear your cache before each test.

[Dan_Sitar]

Thanks guys for your highly valuable inputs to my and many other members' questions. You save our lives.

lucy24, I have created my images in a way that is impossible to misinterpret. They are a text (the name of the image) in a dark color over a light background. Each format of the image has its unique colors as well as its reply:

"image.jpg".jpg , "is_jpg".gif : dark / light turquoise
"image.png".png, "is_png".gif : dark blue / lavender
"image.gif".gif , , "is_gif".gif : brown / antiquewhite
whatever.bla, "is_NOT_image".gif: brown / dark orange

g1smd, I am used to clear my cache before each test and changes I make on my site offline.

I am analyzing all your pointers and I will get back to you. Thank you.

[Dan_Sitar]

Hi, I'm back with a report and new questions...

I spent two days trying to control a third RewriteCond/RewriteRule without success. These two work fine:

RewriteCond %{REQUEST_URI} !"[a-z0-9_-]+\.gif$" [NC]
RewriteCond %{REQUEST_URI} "[a-z0-9_-]+\.jpg$" [NC]
RewriteRule \.jpg$ is_jpg.gif [L]

# "image.jpg".jpg »»» "is_jpg".gif == OK

RewriteCond %{REQUEST_URI} !"[a-z0-9_-]+\.gif$" [NC]
RewriteCond %{REQUEST_URI} "[a-z0-9_-]+\.png$" [NC]
RewriteRule \.png$ is_png.gif [L]

# "image.png".png »»» "is_png".gif == OK

But as soon as I add ONE of those two at the end, it monopolizes the output:

RewriteRule \..* is_NOT_image.gif [L]

# image*.* »»» is_NOT_image.gif == ?

OR

RewriteCond %{REQUEST_URI} !"[a-z0-9_-]+\.jpg$" [NC]
RewriteCond %{REQUEST_URI} !"[a-z0-9_-]+\.png$" [NC]
RewriteCond %{REQUEST_URI} "[a-z0-9_-]+\.gif$" [NC]
RewriteRule \.gif$ is_gif.gif [L]

# image*.* »»» is_gif.gif == ?

Well, I gave up that test and I free you from reading pages of everything that I have tried. So, I decided to go ahead with the real script of my site “subdomain.mydomain.com/”. Note that "community" replaces a well known one. Note that I use "httq" instead of "http" for display purpose.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Part 1, General Hotlinking.

DirectoryIndex index.html
RewriteEngine on

RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?subdomain.mydomain.com/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://(www\.)?mydomain.com/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://(www\.)?google.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://(www\.)?community.com/l.php\?.*$ [NC]
RewriteCond %{HTTP_USER_AGENT} !^communityexternalhit/.*$ [NC]
RewriteRule .*\.(gif|png|jpe?g|eot|woff|svg|ttf|mp3|flv|avi|wmv|mp4|swf|mpe?g|asf|mov)$ animated_promotion_of_my_site.gif [L]

# Great! You wanna steal my stuff? Here's my banner instead!

Three questions:

1. Should I use these flags instead [R=302,L]?

2. Is it safe to combine these two RewriteCond into one?

!^http://(www\.)?subdomain.mydomain.com/.*$
!^http://(www\.)?mydomain.com/.*$
into
!^http://(www\.)?(subdomain\.)?mydomain.com/.*$

3. Should the last RewriteCond %{HTTP_REFERER} ends with:

a. community.com/l.php\?.*$
b. community.com/l.php.*$
c. community.com/.*$

which is this type of query:
httq;//www.community.com/l.php?u=http%3A%2F%2Fsubdomain.mydomain.com%2F&h=.*

By the way, I found this clever hotlinking RewriteRule. Private message me for the reference:

RewriteRule ^(.*)\.(gif|jpe?g|png)$ httq;//%{HTTP_HOST}/path/to/hotlink.php?url=$1.$2 [R,NC,L]

# © Elvis Presley, from his song “Return To Sender”, ha ha ha!

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Part 2, Target Hotlinking.

I want to allow the "community", especially the "community-group" of the dead artist “subdomain”, to post any image and media. But instead, I send only the main photo of the site. It works but with some compromises:

RewriteCond %{HTTP_USER_AGENT} ^communityexternalhit/.*$ [NC]
RewriteRule [^/.]+ mainImage.png [L]

Normally, when I post a link to my site, "community" displays the main photo, the address “subdomain.mydomain.com/” and the description. On the other hand, if I post a link to an image or media of my site, "community" displays the main photo alright, but with The Address Of The Image/Media Instead, e.g., “subdomain.mydomain.com/mainImage.png”. At this point in both cases, the log shows:

"GET /mainImage.png HTTP/1.1" some digits "-" "communityexternalhit/1.0 (+httq://www.community.com/externalhit_uatext.php)"

Two questions:

1. I presume the "_" is the %{HTTP_REFERER} and the "communityexternalhit/.*" is the %{HTTP_USER_AGENT}, right?

2. Is there a way to force "community" to display the address of the site instead of the image/media?

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Part 3, Linking from the posts.

Normally, when I click on the photo or the address (from a post of a link to my site), I get into the site “subdomain.mydomain.com/”. The log shows:

httq;//www.community.com/l.php?u=http%3A%2F%2Fsubdomain.mydomain.com%2F&h=.*

But for a post of an image or media, clicking on them or their addresses open a page with the image alone or my browser cannot open the page for the media. The log shows the same as above plus the image or media. I solved that problem with the following but "community" takes quite some time to figure it out compared to just a link to my site:

RewriteCond %{HTTP_REFERER} ^http://(www\.)?community.com/.*$ [NC]
RewriteRule ^.*$ index.html [L]

Both image and media lead to my index but the URLs in the bar show the same as their posts. We can live with that. But I wonder if that could be fixed by removing the image/media file between the last ‘%2F’ and the ‘&h=’ of the query, like the normal link to the site?

httq;//www.community.com/l.php?u=http%3A%2F%2Fsubdomain.mydomain.com%2FimageMedia.file&h=.*
into
httq;//www.community.com/l.php?u=http%3A%2F%2Fsubdomain.mydomain.com%2F&h=.*

I spent many hours and failed to figure out how to handle a %{QUERY_STRING}/RewriteRule with so many ‘%’ in it that are not back references. Or I just ignore how both servers and the browser deal with a query of another web site.

Question:

How can I manage a %{QUERY_STRING}/RewriteRule with the ‘%’ stuff in the query? I did not find any documentation about such a case.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The entire script:

DirectoryIndex index.html
RewriteEngine on

RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?subdomain.mydomain.com/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://(www\.)?mydomain.com/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://(www\.)?google.*$ [NC]
RewriteCond %{HTTP_USER_AGENT} !^communityexternalhit/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://(www\.)?community.com/l.php\?.*$ [NC]
RewriteRule .*\.(gif|png|jpe?g|eot|woff|svg|ttf|mp3|flv|avi|wmv|mp4|swf|mpe?g|asf|mov)$ animated_promotion_of_my_site.gif [L]

RewriteCond %{HTTP_USER_AGENT} ^communityexternalhit/.*$ [NC]
RewriteRule [^/.]+ mainImage.png [L]

RewriteCond %{HTTP_REFERER} ^http://(www\.)?community.com/.*$ [NC]
RewriteRule ^.*$ index.html [L]

Thank you. Your pointers have been very useful.

[g1smd]

Way too many questions for one post. :) Couple of answers before I have to leave.

Part 1 - Q 2:

!^http://(www\.)?(subdomain\.)?example.com/.*$

combine this as

!^http://(www\.)?(subdomain\.)?example\.com/

(three changes)

Escape ALL literal periods in patterns.

Part 1 - Q 2:

.*$ 

- is redundant on the end of all of the conditions.

Return to sender:

The leading

^(.*)

is an error, use

^(([^/]+/)*[^/.]+)

instead.

Part 2:

.*$ 

- is redundant on the end of all of the conditions.

Part 2 - Q 1:

The HTTP_USER_AGENT variable contains the name of the browser or bot accessing your site.

Part 3:

Escape ALL literal periods in patterns.

.*$ 

- is redundant on the end of all of the conditions.

[lucy24]

What he said ;)

First and simplest: the pattern

RewriteCond %{REQUEST_URI} !"[a-z0-9_-]+\.gif$" [NC]
RewriteCond %{REQUEST_URI} "[a-z0-9_-]+\.jpg$" [NC]

is redundant. If A and B are mutually exclusive, then if something is A, then by definition it is not B. All you need is

%{REQUEST_URI} \.jpg$

... and since all rewrites go to gif, you don't even need that. You only need the one special case where you have to ensure that the request was not "is_gif\.gif"

Conversely, check for request !\.(jpe?g|gif|png)$ to run the "is not image" version. I have deliberately not put it into cut-and-paste form.