Forum Moderators: phranque

Message Too Old, No Replies

Bypassess Image-download blocking code.

.htaccess code

         

essen

9:02 pm on Nov 1, 2015 (gmt 0)

10+ Year Member



RewriteCond %{HTTP_REFERER} !^https://www\.google\.

Not ?

Please assume my regular expression (images|pict/images)/[^/.]+\.(jpe?g|gif|png|ico)|\.(css|js)
is correct. -Not the colprit.

/essen

91.239.168.1 - - [01/Nov/2015:14:33:59 +0100] "GET /images/myImage.jpg" 200 68613 "http://www.google.pl/blank.html" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.80 Safari/537.36"

RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{REQUEST_URI} ^(images|pict/images)/[^/.]+\.(jpe?g|gif|png|ico)|\.(css|js) [NC]
RewriteCond %{HTTP_REFERER} !^https?://example.com [NC]
RewriteCond %{HTTP_REFERER} !^https://www\.google\.
RewriteCond %{HTTP_REFERER} !^https://www\.bing\.com
RewriteCond %{HTTP_REFERER} !^https://www\.zapmeta\. [OR]
RewriteRule !^errorpage403\.html$ - [F,NC]

RewriteCond %{REQUEST_URI} ^((images|pict/images)/[^/.]+\.(jpe?g|gif|png|ico)|\.(css|js)) [NC]
RewriteCond %{HTTP_USER_AGENT} ^facebookexternalhit/. [NC,OR]
RewriteCond %{REQUEST_URI} %1
RewriteCond %{HTTP_REFERER} ^$
RewriteCond %{HTTP_USER_AGENT} !^Googlebot/.
RewriteCond %{HTTP_USER_AGENT} !^bingbot/.
RewriteRule !^errorpage403\.html$ - [F,NC]

essen

9:09 pm on Nov 1, 2015 (gmt 0)

10+ Year Member



Why does this forum skip the begining of my post. This:
Can someone here explain to me why this ip 91.239.168.1
was able to bypass my .htaccess.
The referrer [google.pl...] should be trapped by
RewriteCond %{HTTP_REFERER} !^https://www\.google\.

whitespace

9:58 pm on Nov 1, 2015 (gmt 0)

10+ Year Member Top Contributors Of The Month



Not ?


Exactly.

lucy24

11:12 pm on Nov 1, 2015 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



(images|pict/images)/

Not incorrect, but redundant, since "pict/images" is contained within "images" when there is no opening anchor or preceding content.

Please assume my regular expression (images|pict/images)/[^/.]+\.(jpe?g|gif|png|ico)|\.(css|js)
is correct.

There's no way to say whether a regular expression is correct or incorrect unless you've also specified, in English, what it is intended to do. It's syntactically valid, if that's what you meant, i.e. it won't crash the server. This one means: match anything in the form
images/blahblah.{image-extension-here}
OR
any .css or .js anywhere.

RewriteCond %{HTTP_REFERER} !^https://www\.google\.
<snip>
91.239.168.1 - - [01/Nov/2015:14:33:59 +0100] "GET /images/myImage.jpg" 200 68613 "http://www.google.pl/blank.html" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.80 Safari/537.36"


:: sitting back and twiddling thumbs ::

essen

11:14 pm on Nov 1, 2015 (gmt 0)

10+ Year Member



RewriteCond %{HTTP_REFERER} !^https://www\.zapmeta\. [OR]
[OR] was not there when it was bypasses. It's removed below.

Could you be exact, Whitespace +.



RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{REQUEST_URI} ^(images|pict/images)/[^/.]+\.(jpe?g|gif|png|ico)|\.(css|js) [NC]
RewriteCond %{HTTP_REFERER} !^https?://example.com [NC]
RewriteCond %{HTTP_REFERER} !^https://www\.google\.
RewriteCond %{HTTP_REFERER} !^https://www\.bing\.com
RewriteCond %{HTTP_REFERER} !^https://www\.zapmeta\.
RewriteRule !^errorpage403\.html$ - [F,NC]

RewriteCond %{REQUEST_URI} ^((images|pict/images)/[^/.]+\.(jpe?g|gif|png|ico)|\.(css|js)) [NC]
RewriteCond %{HTTP_USER_AGENT} ^facebookexternalhit/. [NC,OR]
RewriteCond %{REQUEST_URI} %1
RewriteCond %{HTTP_REFERER} ^$
RewriteCond %{HTTP_USER_AGENT} !^Googlebot/.
RewriteCond %{HTTP_USER_AGENT} !^bingbot/.
RewriteRule !^errorpage403\.html$ - [F,NC]

lucy24

11:49 pm on Nov 1, 2015 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



RewriteCond %{HTTP_REFERER} !^https?://example.com [NC]
RewriteCond %{HTTP_REFERER} !^https://www\.google\.

:: continuing to twiddle thumbs, because you are about to kick yourself ::

Incidentally, a RewriteCond involving %{REQUEST_URI} is rarely the most efficient way to do things. Far better to put this part into the body of the rule, so the server doesn't even have to evaluate conditions the rest of the time.

RewriteCond %{REQUEST_URI} %1

Under what circumstances would this condition fail?

essen

12:00 am on Nov 2, 2015 (gmt 0)

10+ Year Member



It's syntactically valid, if that's what you meant, i.e. it won't crash the server.

Of course, that's not what I meant. I have folders "images" and "pict" in the root. And another "images folder" in pict. I do not understand what you're telling me about "Opening anchor". It's absolutely none understandable to me, how pict/images can be under images, - as images/pict/images. Could you show me how it should be instead. It's essential getting it uploaded to my site as soon as possible, although, most of my website images already stolen. There's far too few examples on apache/dochs. And a syntax checker is needed. lyxx and htaccesscheck is not reliable. Neither se I any faultness widt http and https.
That is; the code should bypass [...goo...,...] whereas [...goo......] should be trapped.

/essen

whitespace

12:02 am on Nov 2, 2015 (gmt 0)

10+ Year Member Top Contributors Of The Month



I hadn't just noticed the "https" / "http" difference that lucy24 pointed out (because of the unfortunate way this forum mashes URLs!).

But... the reason for my "Exactly" comment to your "Not ?" query, is that your regex are negated (ie, "not'd"). So it only matches when it "doesn't match" the given regex. Which makes your comment a bit odd...

The referrer [google.pl...] should be trapped by
RewriteCond %{HTTP_REFERER} !^https://www\.google\.


It's not "trapped" by that condition, because it's a negated expression.

Anyway, if your referer is "http://" (which I'd overlooked) then my comment is a bit moot.

essen

12:16 am on Nov 2, 2015 (gmt 0)

10+ Year Member



That is; the code should bypass [...goo...,...] whereas [...goo......] should be trapped.

Now again, this editor Kidding. How can we possibly understand each other when. I come here asking for help, am not prepared for this karma. This editor are loosing donations.
/essen

essen

12:27 am on Nov 2, 2015 (gmt 0)

10+ Year Member



Lucy24
I don't have time for this kidding, while people steals away website resources. I go back to the inferior
apache.../docs

essen

1:30 am on Nov 2, 2015 (gmt 0)

10+ Year Member



Lucy
You should layoff showing a kind of ...
I possibly got it right now - without your helping. example without escaping point
is only existing there - not existing in my real website domain in the uploaded code.
But possibly slashes should be here:
^(/images|/pict/images)/[^/.]+\.(jpe?g|gif|png|ico)|\.(css|js) [NC]
and
%1 be
^%1
No lucy, you're kicking yourself and this newsgroup. Most visitors coming here are grown up people, not class-students. We're having trouble with our websites, and like I said previously, we're not in the mood playing around with you. I'll stay away from here in future. The website holder may delete my account, thanks.
/es...

tangor

2:00 am on Nov 2, 2015 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



We don't give cut and paste answers.

We do provide experience and pointing you in the right direction and suggestions. You might rethink and redo, and take a look at what regex is about (and how it can bite you in the a$$). You had good advice from lucy24 and whitespace.

essen

2:40 pm on Nov 2, 2015 (gmt 0)

10+ Year Member



tangor

It seems some of your guys here are experts insulting OPīs.
Posting a long code didn't ask for a cut and paste. The posted code displayed
a minor error. The answers from your guys - besides being arrogant - render
you've not able to understand plain apache rewriting code. All your talking of
http, https and !^, was complete misunderstood by lucy. The error in the code
doesn't lay there. Did you mention [^/.] in some situations should be [^/\.]. Apache
arenīt a consistence coding language. The docs tells no escaping necessary
inside square brackets. Sometimes it different.

I hope your master soon deletes my account. There's absolutely no donations
after this event with your guys. I'll advice you to grow up as a human having acceptable
behavior to the OPīs, before continua playing arrogant here on behalf of the OPīs.

I looked in here sometimes before. -The same as always.
/es...

whitespace

5:05 pm on Nov 2, 2015 (gmt 0)

10+ Year Member Top Contributors Of The Month



@essen Very sorry that you feel that way, but we are honestly trying to help as best we can. We are only human, we also make mistakes. Helping and learning are, after all, the two main reasons why most of us spend time on these forums. (You mentioned "donations" - we don't get paid btw.)

The error in the code doesn't lay there.


So, where was the error in the code? There are quite a few inconsistencies to know for sure what is and what is not correct.

Looking again at your code, there would seem to be an obvious error in the regex that you told us was "correct. -Not the colprit.". You are missing a slash prefix, so this would never match the request. (?)

Did you mention [^/.] in some situations should be [^/\.]


Do you have an example? AFAIK the dot never needs to be escaped in a character class. It always matches a literal dot. (?)

The docs tells no escaping necessary inside square brackets. Sometimes it different.


Can you reference these docs? I've never seen that stated. Some characters in certain positions do need to be escaped.

lucy24

9:54 pm on Nov 2, 2015 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Some characters in certain positions do need to be escaped.

You get a lot of gratuitous escaping in mod_rewrite. Some of it is because other mods use different syntax, so for example some mods require \/ (the way Javascript does). Some people do seem to take the precautionary measure of "escape everything that isn't an alphanumeric". This isn't actively harmful but does create needless clutter. (Also potential confusion, since forms like "\/" look exactly like "V" in some fonts.)

The one absolutely crucial mod_rewrite escape is the literal space \ and even then it can't be line-final. 80,000 guesses how I know this. Unlike mod_setenvif, you can't override the space-escape requirement by wrapping it all in quotation marks. I suppose someone makes this stuff up.

In any RegEx engine, escaping a . inside a character class would be pretty pointless, since the class by its nature already means "suchandsuch specific characters"; if you meant "any character" you wouldn't use a class. You have to escape \] and \\ but I think even \[ vs. [ depends on RegEx engine.

missing a slash prefix

... and that's why Forums posts always have to be answered by an absolute minimum of two people. What person A misses will be caught by person B, and vice versa. But, as noted above, this part doesn't belong in a RewriteCond anyway; it should be in the body of the rule. I would be inclined to make two rules-- one for image files, one for .css/.js extension-- unless there's a truly vast number of conditions that you don't want to repeat.

whitespace

11:19 pm on Nov 3, 2015 (gmt 0)

10+ Year Member Top Contributors Of The Month



so for example some mods require \/ (the way Javascript does)


I would assume it's only where the slash is used to delimit the regex? Is that not the case? In JavaScript it's just the regex literal that requires the slashes to be escaped, because of the (unchangeable) slash delimiters.

Unlike mod_setenvif, you can't override the space-escape requirement by wrapping it all in quotation marks.


I always thought you could? To be honest, quoting the regex is something I tend to avoid, but my meagre tests seem to work OK?

...this part doesn't belong in a RewriteCond anyway; it should be in the body of the rule.


And then the slash prefix wouldn't be required anyway. (Oh, the irony!)

lucy24

1:37 am on Nov 4, 2015 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I would assume it's only where the slash is used to delimit the regex? Is that not the case?

Probably yes-- in the same way that you don't have to say \/ in javascript if you've used something other than the /blahblah/ construction to set up the regex in the first place. (Matter of fact I can't even remember which Apache mod uses /blahblah/ only that it does exist because I've seen it.)

I always thought you could?


:: detour to test site ::

Damn, you're right. (The "damn" was, ahem, directed at me, not at you.)
RewriteCond %{HTTP_USER_AGENT} " Firefox/[34]"
has the intended result, i.e. the rule it's attached to will execute-- and will not execute if you prepend a ! to the condition. (I was testing in Firefox whatever-number-they're-currently-up-to.)

... and now I'll have to rack my brains trying to figure out what precise combination of circumstances led me to believe that mod_rewrite behaves differently from mod_setenvif in this situation. Bleahh.

:: further detour to docs ::

Oh, right, equals sign. Now, what if you're matching against a query string, so a literal equals sign happens to be the very first thing in your test string? I should probably not think about this too far, or I'll give myself a headache.

:: further memo to self: figure out how to tell Fetch to use Firefox by default, since Camino doesn't approve of IPv6 ::

whitespace

5:39 pm on Nov 4, 2015 (gmt 0)

10+ Year Member Top Contributors Of The Month



Now, what if you're matching against a query string, so a literal equals sign happens to be the very first thing in your test string?


You can escape the equals sign in the CondPattern in this instance to match a literal equals sign.

There is another issue... if you wanted to use the "equals operator" (exact match) on the CondPattern and the string contains a space then AFAIK you have no choice but to use quotes around the entire CondPattern. Attempting to escape the space does not work (I think the slash is seen as a literal slash in this instance). So, the double quotes would seem to be mandatory in the following (hypothetical) example that uses the "equals operator":


RewriteCond %{REQUEST_URI} "=/test/some file.php"