homepage Welcome to WebmasterWorld Guest from 54.242.18.232
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Code, Content, and Presentation / Apache Web Server
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL & phranque

Apache Web Server Forum

    
Strange Problem Resulting from Mod_Rewrite
Wes_0001




msg:1500002
 10:50 pm on Mar 11, 2003 (gmt 0)

Hello, everyone. First post here.

I'm hoping someone can help me with a strange problem resulting from my use of mod_rewrite to block outside linking to my images.

I use (basically) the following in my .htaccess file to bring up a "denied" graphic when someone outside my domain attempts to link to one of my images...


RewriteEngine on
RewriteCond %{HTTP_REFERER}!^http://(www\.)?mydomain.com.*$ [NC]
RewriteRule \.(gif夸pg屹ip如df妃p3妃ov安ma安mv)$ http://mydomain.com/DENIED.GIF [R,L]

This works fine in all appropriate circumstances.

However, my problem derives from my own use of a JavaScript pop-up window to display dynamic content. I call a script that creates a pop-up, then writes links to images (IMG tags) in that pop-up. It works wonderfully in all of the most common browsers except NN4.

When it opens in NN4, the pop-up apparently sends no referer when requesting the images, so all the images are replaced with that "denied" graphic mentioned above.

What I'm hoping I can do is modify my use of mod_rewrite to serve the "denied" graphic whenever an image is called from outside my domain EXCEPT if there is no referer and the user_agent is that of NN4.

I've been round and round with this, but can't seem to get it right.

Incidentally -- and understand I'm not that adept at this sort of stuff -- I tried the following with no success:


RewriteEngine on

RewriteCond %{HTTP_REFERER} ^$ [C]
# If the referer is empty... (If it's not, skip the next rule because it's [C]hained.)
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/4.*$ [S=2]
# ...and if the user_agent is NN 4.x, then skip the next 2 rules, effectively ending the sequence.
# Otherwise, continue.

RewriteCond %{HTTP_REFERER}!^http://(www\.)?mydomain.com.*$ [NC]
# If the referer doesn't match the URL above...
RewriteRule \.(gif夸pg屹ip如df)$ http://mydomain.com/DENIED.GIF [R,L]
# ...then return the "denied" graphic.

All I get was a server error when trying to access the site at all, from any browser.

Any ideas?

 

andreasfriedrich




msg:1500003
 12:47 am on Mar 12, 2003 (gmt 0)

C and S are invalid flags for RewriteCond [httpd.apache.org] rules. The RewriteCond [httpd.apache.org] directive allows only NC and OR.

>>serve the "denied" graphic whenever an image is called
>>from outside my domain EXCEPT if there is no referer
>>and the user_agent is that of NN4

You would need to write something like this:

RewriteCond [httpd.apache.org] %{HTTP_REFERER}__SPACE__!^http://(www\.)?mydomain.com.*$ [NC]
RewriteCond [httpd.apache.org] %{HTTP_USER_AGENT}<->%{HTTP_REFERER}__SPACE__!^NS4_UA_String<->$
RewriteRule [httpd.apache.org] \.(gif夸pg屹ip如df)$ [mydomain.com...] [L]

It愀 rather later and I惴 not thinking too clearly now. Can愒 seem to recall anything I ever learned on logic (not domain_ref and not (ns4 and empty ref) can be rewritten as?). But I do believe this to work. I just tested this on a piece of paper.

HTH Andreas

jdMorgan




msg:1500004
 12:56 am on Mar 12, 2003 (gmt 0)

Many ua's will provide no referrer, especially if the visitor is using Norton Internet Security. The following ruleset will fix bith problems.


RewriteEngine on
RewriteCond %{HTTP_REFERER}!^$
RewriteCond %{HTTP_REFERER}!^http://(www\.)?mydomain.com [NC]
RewriteRule \.(gif夸pg屹ip如df妃p3妃ov安ma安mv)$ /DENIED.GIF [L]

Since we're in logic mode, this reads, "If the referer is not blank AND the referer is not my domain, then redirect all image requests transparently to the replacement image, and stop processing rewrite rules."
Simple is good. :)

I'd be interested to hear confirmation that redirecting a video file to a .gif image really works.

Jim

andreasfriedrich




msg:1500005
 1:01 am on Mar 12, 2003 (gmt 0)

I thought about adding that advise about empty referrers but then forgot. I guess it is too late for me ;). Thanks for adding that Jim.

Andreas

jdMorgan




msg:1500006
 1:11 am on Mar 12, 2003 (gmt 0)

Andreas,

Time for some rest! :)

But now I am starting to understand the <-> concatenation operator you are so fond of.

Jim

Wes_0001




msg:1500007
 2:29 am on Mar 16, 2003 (gmt 0)

Thanks for the help, folks. Unfortunately, this approach just ain't working for me. I think I'm just going to have to test for the browser type in my JavaScript and deliver the information differently for NN4.

Oh, and BTW, the newest version of Windows Media Player will display an image when redirected. However, I had included the other file types only as a temporary solution, which I am about to change.

Thanks again for the help.

jdMorgan




msg:1500008
 2:43 am on Mar 16, 2003 (gmt 0)

...Dang posting software ate my spaces again - try it this way:

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?mydomain.com [NC]
RewriteRule \.(gif夸pg屹ip如df妃p3妃ov安ma安mv)$ /DENIED.GIF [L]

You must also replace the broken vertical pipes (¦) with solid ones.

Don't give up so easily - most of us have this working with pretty much the same code.
My only reservation is that some browsers may not accept a redirect from a jog to a gif, so I always replace each denied image file type with its own "denied" image, using:

RewriteRule \.(gif夸pg屹ip如df妃p3妃ov安ma安mv)$ /DENIED.$1 [L]

Jim

Wes_0001




msg:1500009
 5:28 am on Mar 16, 2003 (gmt 0)

Oh, please don't mistake this for giving up easily. I've been battling this thing for quite some time now. You guys are my last hope. :)

Unfortunately, "RewriteCond %{HTTP_REFERER} !^$" doesn't make a difference. In fact, its actually in my working list of RewriteConds.

I believe the above basically says, "If the referer is not blank (then redirect the image to DENIED.GIF)." But my problem is that the referer is blank, apparently, and it's still redirecting it.

Incidentally, every example I've seen so far of the RewriteRule includes both the R and L flags, but I noticed that you only use the L. Any reason?

jdMorgan




msg:1500010
 1:52 pm on Mar 16, 2003 (gmt 0)

Wes_001,

If you use a canonical URL beginning with "http:" and an [R] flag in a RewriteRule, then the redirect will be external. A 301 or 302 response code will be sent back to the user-agent (browser) along with the new resource (in this case, the /DENIED.GIF image) URL. The browser will re-issue the request using the new URL, and your mod_rewrite rules will be re-invoked for this new request. This will lead to an infinite loop unless you add another RewriteCond to prevent it. (The only reason the code with an [R} flag does not give you an infinite loop right now is that the substitute image name is uppercase, and so does not match the RewriteRule.)

Alternately, if a local path is used instead of a canonical URL, and the [R] flag is left out, the server simply substitutes the /DENIED.GIF image for whatever was requested in the initial request.

So, the omission of [R] is non-trivial in the example code I posted above.

One thing I should mention... WebmasterWorld member guabito discovered a few bad-bots were using a user agent which consisted of a single hyphen character "-". Not blank, but actually a single hyphen. This was a "visual trick," since most raw logs will show a "-" for a blank referrer, so this hyphen UA looked just like a blank UA in the logs, except that the rewrites were not invoked. Using a single hyphen as a user-agent name was a trick to bypass blank referrer checks. This is not what is causing your problem, but it's worht mentioning.

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?mydomain\.com [NC]
RewriteRule \.(gif夸pg屹ip如df妃p3妃ov安ma安mv)$ /DENIED.GIF [L]

This code snippet should work fine, as long as the broken vertical pipe characters are edited to use solid ones, your domain name is substituted into the second line correctly, and everything else is exactly as shown. If the code does not work after that, then something else is wrong - possibly interference from other (preceding) rewrite rules, or a server config problem.

Jim

Wes_0001




msg:1500011
 7:17 pm on Mar 16, 2003 (gmt 0)

Please understand that my redirect is working fine. Any links to images from outside my domain result in the denied image.

The only place I'm having a problem is when I dynamically create content in a pop-up window (using document.write), and only in NN4. All images in the pop-up result in the denied image. There's something else going on here, and the only explanation I can come up with is that there is no referer being delivered by the pop-up in NN4.

I'm trying to prevent the redirect from occuring when there is no referer and when NN4 is the UA.

The code you offered is what I have and it doesn't prevent the problem.

jdMorgan




msg:1500012
 7:48 pm on Mar 16, 2003 (gmt 0)

OK,

I assume from what you've written that you do not have access to raw log files to see what the user-agent is in these cases. Having pondered your problem a bit, I recalled discussing a similar problem with another poster here at WebmasterWorld, and - lucky you - I still had an archived stickymail from him.

There is some strange problem with NN4 (or older) using JS pops that causes it to use the Netscape Composer User-Agent "WYSIWYG" when requesting images. I'm not sure if I have this exactly right, and it would be better to check it by looking at raw logs, but try this:

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^(wysiwyg:)?http://(www\.)?mydomain\.com [NC]
RewriteRule \.(gif夸pg屹ip如df妃p3妃ov安ma安mv)$ /DENIED.GIF [L]

My memory ain't what it used to be, so if anybody else remembers this odd NN4 behaviour and can provide more info, I'd appreciate it.

HTH,
Jim

Wes_0001




msg:1500013
 10:52 pm on Mar 16, 2003 (gmt 0)

Woohoo!

You were right about it adding "wysiwyg" to the beginning. A little more research revealed it's actually "wysiwyg://[number]/" so the following appears to do the trick:


RewriteCond %{HTTP_REFERER} !^(wysiwyg:.*)?http://(www\.)?mydomain.com.*$ [NC]

I can finally implement my all-in-one pop-up script!

Thanks a bunch for the help!

jdMorgan




msg:1500014
 1:00 am on Mar 17, 2003 (gmt 0)

Hmm...

I don't personally like using .* like that - it will accept any number of any characters there - and is slow.

How about using a "class" to accept just the digits 0-9, there?

RewriteCond %{HTTP_REFERER}!^(wysiwyg:[0-9])?http://(www\.)?mydomain.com.*$ [NC]

or you could use "[0-9]{1,2}" in that spot if there might be one or two digits present.

Glad it worked!
Jim

Wes_0001




msg:1500015
 1:04 am on Mar 17, 2003 (gmt 0)

Ah, that's how I would do that.

Yeah, I found an example someone used in which the number was 11, so I would go with [0-9]{1,2}.

Thanks again.

zollerwagner




msg:1500016
 7:49 pm on Mar 19, 2003 (gmt 0)

A similar problem on my site occurred when I try to access images on document.write-created pages. Internet explorer has no problem--as far as I know--but some version of Netscape add characters to the beginning of the referrer value.

Unfortunately, the characters differ, depending on the browser and platform. I don't have access to a wide variety of these for testing, but I do know this:

Netscape 4.75 on PC adds something like: wysiwyg://0/
but Netscape 7.0 on Mac adds something like: wyciwyg://0/

The difference is a "s" or "c". The numbers may vary, too.

Our current solution is:


# for some versions of Netscape, must add a condition to allow images on document.write-created pages
# (wysiwgy://[0-9]*/)? for netscape 4.75 on PC (Mac?)
# (wyciwyg://[0-9]*/)? for Netscape 7 (others?) on Mac
RewriteCond %{HTTP_REFERER}!^(wy[sc]iwyg://[0-9]*/)?http://(www\.)?DomainName\.com [NC]

Remember that this forum changes things about "" and "!". You'll need to add a space before the "!". I don't think there's a "" here, but if there were, you'd have to change it to one without the gap in the middle.

You can take out the comments, but I have to put the comments in to remind myself of this craziness. Otherwise, I'll forget everything in between edits.

I owe a large debt to jdMorgan, who taught me how investigate the causes and helped me come up with this answer. Thanks, Jim!

Wes_0001




msg:1500017
 8:16 pm on Mar 19, 2003 (gmt 0)

Excellent addition, zoller. Thanks for adding that.

BTW, I don't know if you copied that straight from your .htaccess, but if you did, you've got a typo in there. (Of course, it's just a comment, but it would bug me. :)


# for some versions of Netscape, must add a condition to allow images on document.write-created pages
# ([b]wysiwgy[/b]://[0-9]*/)? for netscape 4.75 on PC (Mac?)
# (wyciwyg://[0-9]*/)? for Netscape 7 (others?) on Mac
RewriteCond %{HTTP_REFERER}!^(wy[sc]iwyg://[0-9]*/)?http://(www\.)?DomainName\.com [NC]

zollerwagner




msg:1500018
 9:00 pm on Mar 19, 2003 (gmt 0)

Ah, you're right, Wes! Thanks for picking that up.

I'd also be interested in hearing if anyone finds that there are other modifications needed for other versions of Mozilla or Netscape.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved