homepage Welcome to WebmasterWorld Guest from 54.227.5.234
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor
Visit PubCon.com
Home / Forums Index / Code, Content, and Presentation / Apache Web Server
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL & phranque

Apache Web Server Forum

    
How are you blocking download managers these days
htaccess download manager block
hottrout




msg:4549621
 11:29 pm on Feb 27, 2013 (gmt 0)

A few years ago I added the following code to my htaccess

# Prevent Download managers from being used
RewriteCond %{HTTP_USER_AGENT} ^asterias [OR]
RewriteCond %{HTTP_USER_AGENT} ^BackDoorBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^Black.Hole [OR]
RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR]
RewriteCond %{HTTP_USER_AGENT} ^BlowFish [OR]
RewriteCond %{HTTP_USER_AGENT} ^BotALot [OR]
RewriteCond %{HTTP_USER_AGENT} "bsalsa" [OR]
RewriteCond %{HTTP_USER_AGENT} ^BuiltBotTough [OR]
RewriteCond %{HTTP_USER_AGENT} ^Bullseye [OR]
RewriteCond %{HTTP_USER_AGENT} ^BunnySlippers [OR]
RewriteCond %{HTTP_USER_AGENT} ^Cegbfeieh [OR]
RewriteCond %{HTTP_USER_AGENT} ^CheeseBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^CherryPicker [OR]
RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR]
RewriteCond %{HTTP_USER_AGENT} ^CopyRightCheck [OR]
RewriteCond %{HTTP_USER_AGENT} ^cosmos [OR]
RewriteCond %{HTTP_USER_AGENT} ^Crescent [OR]
RewriteCond %{HTTP_USER_AGENT} ^Custo [OR]
RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR]
RewriteCond %{HTTP_USER_AGENT} ^DittoSpyder [OR]
RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR]
RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR]
RewriteCond %{HTTP_USER_AGENT} ^EirGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailCollector [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR]
RewriteCond %{HTTP_USER_AGENT} ^EroCrawler [OR]
RewriteCond %{HTTP_USER_AGENT} ^Express\ WebPictures [OR]
RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR]
RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE [OR]
RewriteCond %{HTTP_USER_AGENT} ^FDM [OR]
RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^FlipDog [OR]
RewriteCond %{HTTP_USER_AGENT} ^Foobot [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetWeb! [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR]
RewriteCond %{HTTP_USER_AGENT} ^GrabNet [OR]
RewriteCond %{HTTP_USER_AGENT} ^Grafula [OR]
RewriteCond %{HTTP_USER_AGENT} ^Harvest [OR]
RewriteCond %{HTTP_USER_AGENT} ^hloader [OR]
RewriteCond %{HTTP_USER_AGENT} ^HMView [OR]
RewriteCond %{HTTP_USER_AGENT} ^httplib [OR]
RewriteCond %{HTTP_USER_AGENT} "HTTrack" [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^humanlinks [OR]
RewriteCond %{HTTP_USER_AGENT} ^ia_archiver [OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Stripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} ^Indy\ Library [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^InfoNaviRobot [OR]
RewriteCond %{HTTP_USER_AGENT} ^InterGET [OR]
RewriteCond %{HTTP_USER_AGENT} ^Internet\ Ninja [OR]
RewriteCond %{HTTP_USER_AGENT} ^Irvine [OR]
RewriteCond %{HTTP_USER_AGENT} ^JennyBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^JetCar [OR]
RewriteCond %{HTTP_USER_AGENT} ^JOC\ Web\ Spider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Kenjin.Spider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Keyword.Density [OR]
RewriteCond %{HTTP_USER_AGENT} ^larbin [OR]
RewriteCond %{HTTP_USER_AGENT} ^LeechFTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^LexiBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^libWeb/clsHTTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^LinkextractorPro [OR]
RewriteCond %{HTTP_USER_AGENT} ^LinkScan/8.1a.Unix [OR]
RewriteCond %{HTTP_USER_AGENT} ^LinkWalker [OR]
RewriteCond %{HTTP_USER_AGENT} ^lwp-trivial [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mass\ Downloader [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mata.Hari [OR]
RewriteCond %{HTTP_USER_AGENT} ^MIDown\ tool [OR]
RewriteCond %{HTTP_USER_AGENT} ^MIIxpc [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mister.PiX [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mister\ PiX [OR]
RewriteCond %{HTTP_USER_AGENT} ^moget [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/3.Mozilla/2.01 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla.*NEWT [OR]
RewriteCond %{HTTP_USER_AGENT} ^Navroad [OR]
RewriteCond %{HTTP_USER_AGENT} ^NearSite [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetAnts [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetMechanic [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetZIP [OR]
RewriteCond %{HTTP_USER_AGENT} ^NICErsPRO [OR]
RewriteCond %{HTTP_USER_AGENT} ^NPBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^Octopus [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline.Explorer [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Navigator [OR]
RewriteCond %{HTTP_USER_AGENT} ^Openfind [OR]
RewriteCond %{HTTP_USER_AGENT} ^PageGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^PuxaRapido [OR]
RewriteCond %{HTTP_USER_AGENT} ^Papa\ Foto [OR]
RewriteCond %{HTTP_USER_AGENT} ^pavuk [OR]
RewriteCond %{HTTP_USER_AGENT} ^pcBrowser [OR]
RewriteCond %{HTTP_USER_AGENT} ^ProPowerBot/2.14 [OR]
RewriteCond %{HTTP_USER_AGENT} ^ProWebWalker [OR]
RewriteCond %{HTTP_USER_AGENT} ^ProWebWalker [OR]
RewriteCond %{HTTP_USER_AGENT} ^ReGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^RepoMonkey [OR]
RewriteCond %{HTTP_USER_AGENT} ^RMA [OR]
RewriteCond %{HTTP_USER_AGENT} ^SiteSnagger [OR]
RewriteCond %{HTTP_USER_AGENT} ^SiteSucker [OR]
RewriteCond %{HTTP_USER_AGENT} ^Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} ^SlySearch [OR]
RewriteCond %{HTTP_USER_AGENT} ^SmartDownload [OR]
RewriteCond %{HTTP_USER_AGENT} ^SpankBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^spanner [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Surfbot [OR]
RewriteCond %{HTTP_USER_AGENT} ^suzuran [OR]
RewriteCond %{HTTP_USER_AGENT} ^Szukacz/1.4 [OR]
RewriteCond %{HTTP_USER_AGENT} ^tAkeOut [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro [OR]
RewriteCond %{HTTP_USER_AGENT} ^Telesoft [OR]
RewriteCond %{HTTP_USER_AGENT} ^The.Intraformant [OR]
RewriteCond %{HTTP_USER_AGENT} ^TheNomad [OR]
RewriteCond %{HTTP_USER_AGENT} ^TightTwatBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^Titan [OR]
RewriteCond %{HTTP_USER_AGENT} ^toCrawl/UrlDispatcher [OR]
RewriteCond %{HTTP_USER_AGENT} ^toCrawl/UrlDispatcher [OR]
RewriteCond %{HTTP_USER_AGENT} ^True_Robot [OR]
RewriteCond %{HTTP_USER_AGENT} ^turingos [OR]
RewriteCond %{HTTP_USER_AGENT} ^TurnitinBot/1.5 [OR]
RewriteCond %{HTTP_USER_AGENT} ^URLy.Warning [OR]
RewriteCond %{HTTP_USER_AGENT} ^VCI [OR]
RewriteCond %{HTTP_USER_AGENT} ^VoidEYE [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebAuto [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebBandit [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebEMailExtrac.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebEnhancer [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebFetch [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebGo\ IS [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web.Image.Collector [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Image\ Collector [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebLeacher [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebmasterWorldForumBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebSauger [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ eXtractor [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website.Quester [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ Quester [OR]
RewriteCond %{HTTP_USER_AGENT} ^Webster.Pro [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebWhacker [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZip [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget [OR]
RewriteCond %{HTTP_USER_AGENT} ^Widow [OR]
RewriteCond %{HTTP_USER_AGENT} ^[Ww]eb[Bb]andit [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWW-Collector-E [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xenu's [OR]
RewriteCond %{HTTP_USER_AGENT} ^Zeus
RewriteRule .* - [F]

I also read another interesting approach to blocking spambots and crawlers at this thread but was not sure if this would prevent download managers scraping my site.

[webmasterworld.com...]

Can anyone reccomend a better way to prevent download managers or is this still the way people do it. If it is still the way, is my list up to date?

 

wilderness




msg:4549643
 1:56 am on Feb 28, 2013 (gmt 0)

How like in method of Apache procedure?

Either mod_authz or mod_rewrite.

As to your decade old copy and pasted list and some logic?
This thread belongs in the SSID forum.

lucy24




msg:4549679
 5:13 am on Feb 28, 2013 (gmt 0)

This may just be me, but rules involving long strings of [OR] conditions make me nervous. If the options are too long to fit in a single pipe-delimited list, handle them a different way.

Oh, and your entire rule would fail disastrously because you've got an opening anchor on every single User-Agent. Few robots are thoughtful enough to put their most important element right at the beginning.

hottrout




msg:4549772
 9:15 am on Feb 28, 2013 (gmt 0)

HotTrout sits down and takes a long sip of coffee.


I guess this is utter crap then. Do people at least still use USER_AGENT names to catch download managers or is there some other better way. I think I am going to now remove this entirly from my htaccess.

Lucy you pointed out that the statements open with ^ and I also thought that was odd but every example I have seen used this method. I did change the HTTrack to be inside quotes because I noticed that it never started HTTRack but the others I just left as they were.

wilderness




msg:4549792
 11:30 am on Feb 28, 2013 (gmt 0)

Lucy you pointed out that the statements open with ^ and I also thought that was odd but every example I have seen used this method.


It was not unusual for a decade ago, in fact it was the norm.

lucy24




msg:4549798
 11:49 am on Feb 28, 2013 (gmt 0)

I don't think you ever need quotation marks in mod_rewrite. You sometimes need them in mod_setenvif-- which is where I've got my equivalent rule-- if your text block contains a literal space. (Hm, come to think of it, could you escape the space instead? That's what you'd do in a RewriteRule.)

The mod_authz version works in conjunction with mod_setenvif. First you list your unwanted UAs:

BrowserMatch Clipish keep_out
BrowserMatch Covario keep_out
BrowserMatch Slurp keep_out

et cetera

And then when you get to your Deny from directives-- the same place you list all the unwanted IP ranges-- you throw in a sweeping

Deny from env=keep_out

It was not unusual for a decade ago, in fact it was the norm.

Were User Agent strings differently arranged back then, or was it simply the norm for rules not to work? :) It can't possibly be a change in Regular Expressions; they're almost as carved-in-stone as the Unicode consortium.

wilderness




msg:4549804
 12:11 pm on Feb 28, 2013 (gmt 0)

I don't think you ever need quotation marks in mod_rewrite.


That is correct, despite what the Apache Docs provide.

Were User Agent strings differently arranged back then


Yes.

wilderness




msg:4549814
 12:22 pm on Feb 28, 2013 (gmt 0)

Were User Agent strings differently arranged back then


Yes


lucy,
My apologies.
I should expand on this.

The compliant browser UA's (begins with Mozilla) were quite similar to those currently used.

The harvester/download (aka kiddie tools) were much more abundant, and they did not use the compliant browsers UA's.
That's what made them so easy to stop.

lucy24




msg:4549993
 12:10 am on Mar 1, 2013 (gmt 0)

The compliant browser UA's (begins with Mozilla)

Or "Opera" as the case may be ;) Far as I can make out, desktop Operas start with "Mozilla" but assorted mobiles-- notably Android-- start with "Opera 9.80".

dougwilson




msg:4550007
 1:55 am on Mar 1, 2013 (gmt 0)

Is it March already?

Recently cleaned my htaccess. Not really blocking much anymore. But I still watch and below are some user agents and requests that annoyed me enough to add to server process time. Yes I use "" for strings and ^ sometimes. Checking http viewer (rex swain) for each.

# bots naughty or no return
SetEnvIfNoCase User-agent ^\xff blocked
SetEnvIfNoCase User-agent Curl blocked
SetEnvIfNoCase User-agent ".exe" blocked
SetEnvIfNoCase User-agent HTTrack blocked
SetEnvIfNoCase User-agent magpie-crawler blocked
SetEnvIfNoCase User-agent "Web Core / Roots" blocked
SetEnvIfNoCase user-agent "MSIE\ 5.0" blocked

# requests
SetEnvIfNoCase Request_URI //db blocked
SetEnvIfNoCase Request_URI /user/soapCaller.bs blocked
deny from env=blocked

Most of this (above) is just the difference between a 403 and a 404.

What I am serious about denying are legit data mining outfits - because I despise them.

lucy24




msg:4550010
 2:25 am on Mar 1, 2013 (gmt 0)

"MSIE\ 5.0"


Mine says "MSIE [1-4]\." The quotation-marks-and-escaping combination is, uhm, belt and suspenders. Or belt and braces, depending on where you live. Will probably do no harm, but isn't needed. But if you do escape a literal space, you should escape literal periods as well.

The trailing period was, ahem, added after the release of MSIE 10. MSIE 5 exists in versions other than .0 but I've got some exemptions involving the extreme north of Canada, so 5 and 6 are handled in mod_rewrite.

dougwilson




msg:4550026
 4:37 am on Mar 1, 2013 (gmt 0)

I was seriously looking at some suspenders the other day. Of course you're right. These are just some examples of user agents & requests seen in logs. Key being, for questioner, in logs.

hottrout




msg:4550069
 10:16 am on Mar 1, 2013 (gmt 0)

Just to make sure I am correct in my own code before using it, Will this structure work in the htaccess in terms of position and code :-

RewriteEngine On

SetEnvIfNoCase User-agent (Access|appid) blocklist
SetEnvIfNoCase User-agent (Capture|Client|Copy|crawl|curl) blocklist
SetEnvIfNoCase Request_URI //db blocklist
SetEnvIfNoCase Request_URI /user/soapCaller.bs blocklist
SetEnvIf User-Agent " ; " blocklist
SetEnvIf User-Agent " \( " blocklist
SetEnvIf User-Agent "; " blocklist

order allow,deny
deny from blocklist
allow from all

Is the difference between SetEnvIf and SetEnvIfNoCase that the second just ignores the case?

lucy24




msg:4550081
 10:35 am on Mar 1, 2013 (gmt 0)

If you're blocking spoofers, NoCase is good because it covers more forms. If "yukbot" is bad, then an imitation "YukBot" is just as bad. But if you're allowing good guys, NoCase is bad because it might let in people who get it wrong-- claiming to be "googlebot" f'rinstance.

Did you really mean to block UA strings containing semicolon followed by a space? Isn't that the normal configuration? As in "{blahblah} (compatible; {morestuff})" et cetera.

I wanted to block UAs containing two consecutive spaces-- like the plainclothes bingbot-- and then I met a legitimate human UA in that form. Or was it a space before a semicolon? I forget. Dang. Every time you think you've found a pattern that can only be a robot, up crops a human in the identical clothes.

BrowserMatch or BrowserMatchNoCase is a useful shorthand within mod_setenvif.

wilderness




msg:4550093
 11:46 am on Mar 1, 2013 (gmt 0)

Did you really mean to block UA strings containing semicolon followed by a space?


The forum software removed the second space

";  "

[edited by: engine at 3:28 pm (utc) on Mar 1, 2013]

lucy24




msg:4550100
 12:35 pm on Mar 1, 2013 (gmt 0)

Ah, semicolon plus two spaces, mark of the plainclothes bingbot ;)

I thought I was making that up, but quick riffle through last few days' raw logs, search for ";  " (second one's an nbsp) turns up nothing but the occasional spurt of

131.253.blahblah
and
65.55.blahblah

Those are blocked with mod_rewrite because I'm using the UA-plus-IP combination.

Hm. Interesting chiasmus there. "Google UA from non-Google IP, or non-bingbot UA from Bing IP". Other way around, not so much.

dougwilson




msg:4550142
 3:16 pm on Mar 1, 2013 (gmt 0)

I've used: A test blog, or folder with one page and an htaccess file in it. Then use rex swains http viewer to check if rules are working as I intend.

Test lots of legitimate UA's to see if they get through too.

But, there's no law saying SiteCopy has to continue as SiteCopy when it stops getting access. SiteCopy can decide to use myScrapy as a UA.

If the people we want to block are coming from ISP's there's little I've found I can do about it, unless I want to block 100's or thousands of IP's that might change tomorrow.

Change, as in, Bob's not using this IP anymore Jane is, and Jane is now being blocked.

I read a lot online where the Baby gets tossed with the Bath Water

wilderness




msg:4550162
 4:08 pm on Mar 1, 2013 (gmt 0)

This thread still belongs in the SSID forum.

Leosghost




msg:4550171
 4:45 pm on Mar 1, 2013 (gmt 0)

This thread still belongs in the SSID forum.

The thing is ..a newbie ( which we all were once :) ..would n't know to ask in the SSID forum about this..and if s/he ( or anyone else, particularly "newbies" reading ) has bookmarked this page, moving it at this stage would lose them..

wilderness




msg:4550179
 5:24 pm on Mar 1, 2013 (gmt 0)

Leo,
With all due respect, and to the moderator on an editing binge as well.

I've been a participating member here at Webmaster World three years prior to yourself.

Do you honestly believe I need an explanation of the functionality of Webmaster World (rhetorical).

This crap (the original inquiry has been pasted into Webmaster World forums thousands of times.

Providing the same explanations and making the sane editing corrections that were required more than a decade ago are of no benefit to anybody, not even the noob making the inquiry.

Leosghost




msg:4550183
 5:38 pm on Mar 1, 2013 (gmt 0)

Do you honestly believe I need an explanation of the functionality of Webmaster World (rhetorical).

apparently so..*
I've been a participating member here at Webmaster World three years prior to yourself.

who has been here longest is not relevant..

I repeat..
The thing is ..a newbie ( which we all were once :) ..would n't know to ask in the SSID forum about this..
So why get on the newbie's or the mod's case about it..

*Ah..I get it ..you were born knowing all this stuff, you never had to ask..always knew exactly the right forums and threads to ask questions in..and to know who benefits from reading what..must be cool being omniscient and omnipotent..the Vatican has a vacancy..quick :)

hottrout




msg:4550185
 5:46 pm on Mar 1, 2013 (gmt 0)

Please dont turn this into a griefing session. The fact is that wilderness was very helpful to me and for that I am grateful. I am a noob but not devoid of brains and to be honest I was not aware of the SSID, I am now. I just thought that because my normal questions are around the htaccess file that they belonged in this forum.

Sorry to have caused friction.

Leosghost




msg:4550189
 5:51 pm on Mar 1, 2013 (gmt 0)

Not your fault hottrout..
I am a noob but not devoid of brains and to be honest I was not aware of the SSID, I am now. I just thought that because my normal questions are around the htaccess file that they belonged in this forum.

I rest my case :)

wilderness




msg:4550190
 6:01 pm on Mar 1, 2013 (gmt 0)

the Vatican has a vacancy..quick


If religion and politics is now permissible in the Apache Forum, than editing moderators are going to have their heyday when all the hogwash expands beyond their wildest imaginations.

Further more, If religion and politics are now permissible in the Apache Forum, thus is profanity!

wilderness




msg:4550191
 6:02 pm on Mar 1, 2013 (gmt 0)

I rest my case


And apparently you are just as void at locating the forum charter.

SevenCubed




msg:4550193
 6:13 pm on Mar 1, 2013 (gmt 0)

Eeeeeeek fire in aisle Apache < cube running with pitcher of cold water...splash! >...group hug guys, chill. Please.

ergophobe




msg:4550204
 6:50 pm on Mar 1, 2013 (gmt 0)

If the options are too long to fit in a single pipe-delimited list


A little OT, but Lucy I thought I would pick up on this. I know you're using that as a rule of thumb, not advocating a piped regex over the [OR] statements, but for other people who might be reading that, I thought I would mention this...

In general, chained RewriteConds using [OR] perform better than a regex as benchmarked many years ago by Andreas Friedriech.

Now, many years ago may be a problem, since it's quite possible that mod_rewrite has refactored how it does regex searches. I don't know as anyone has tested it recently.

lucy24




msg:4550255
 10:03 pm on Mar 1, 2013 (gmt 0)

In general, chained RewriteConds using [OR] perform better than a regex as benchmarked many years ago by Andreas Friedriech.

That's really not surprising. The essence of an [OR] delimited list is that the moment one condition is met, you can skip to the end of the [OR] cluster. But with a pipe-delimited list you still have to fast-forward to the closing parenthesis-- assuming there was an opening one --which is not so easy if you are a computer operating in one dimension.

Especially when all this is happening in htaccess.

In the case of RewriteRule directives, in .htaccess context these regular expressions must be re-compiled with every request to the directory, whereas in main server configuration context they are compiled once and cached.

I just recently met this line. I may have met it earlier and forgot, so I am trying hard to remember it this time.

But you probably realize I'm thinking from the human user's POV. One slip in cutting-and-pasting or rearranging lines, and your RewriteRule with [OR] conditions brings the whole site crashing down. mod_rewrite will not forgive a trailing [OR] at the end of the last Condition. (It probably won't forgive a trailing pipe creating a null group either, but it's more noticeable.)

It's like minifying. OK, you save a few bytes, but the whole thing becomes unreadable. Besides, the word "minify" makes my skin crawl.




I think everyone who posts regularly in the Apache forum has their own particular boilerplate. Don's happens to be "This discussion belongs in SSID." ;)

wilderness




msg:4550306
 2:24 am on Mar 2, 2013 (gmt 0)

Don belongs in the SSID forum, henceforth!

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved