Welcome to WebmasterWorld Guest from 54.234.153.186

Forum Moderators: ergophobe

Message Too Old, No Replies

Replacement .htaccess files for Joomla 1.5 and Joomla 1.6

Large speed boost gained by optimising the mod_rewrite code.

     

g1smd

11:48 am on Mar 1, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



The default
.htaccess
file supplied with Joomla 1.5 and Joomla 1.6 has a number of deficiencies and inefficiencies.

Below, replacement files, to fix these issues.

See also:
[joomlacode.org...] (Joomla 1.5)
[joomlacode.org...] (Joomla 1.6)

g1smd

11:51 am on Mar 1, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Joomla 1.5

##
# @version $Id: BASED ON htaccess.txt 14401 2010-01-26 14:10:00Z louis $
# @MODIFIED 2011-02-25
# @package Joomla
# @copyright Copyright (C) 2005 - 2010 Open Source Matters. All rights reserved.
# @license http://www.gnu.org/copyleft/gpl.html GNU/GPL
# Joomla! is Free Software
##


#####################################################
# READ THIS COMPLETELY IF YOU CHOOSE TO USE THIS FILE
#
# The line just below this section: 'Options +FollowSymLinks' may cause problems
# with some server configurations. It is required for use of mod_rewrite, but may already
# be set by your server administrator in a way that dissallows changing it in
# your .htaccess file. If using it causes your server to error out, comment it out (add # to
# beginning of line), reload your site in your browser and test your sef url's. If they work,
# it has been set by your server administrator and you do not need it set here.
#
#####################################################

## Can be commented out if causes errors, see notes above.
Options +FollowSymLinks

#
# mod_rewrite in use

RewriteEngine On

########## Begin - Rewrite rules to block out some common exploits
## If you experience problems on your site block out the operations listed below
## This attempts to block the most common type of exploit `attempts` to Joomla!
#
## Deny access to extension xml files (uncomment out to activate)
#<Files ~ "\.xml$">
#Order allow,deny
#Deny from all
#Satisfy all
#</Files>
## End of deny access to extension xml files
# Block out any script trying to set a mosConfig value through the URL
RewriteCond %{QUERY_STRING} mosConfig_[a-zA-Z_]{1,21}(=|\%3D) [OR]
# Block out any script trying to base64_encode data within the URL
RewriteCond %{QUERY_STRING} base64_encode[^(]*\([^)]*\) [OR]
# Block out any script that includes a <script> tag in URL
RewriteCond %{QUERY_STRING} (<|%3C)([^s]*s)+cript.*(>|%3E) [NC,OR]
# Block out any script trying to set a PHP GLOBALS variable via URL
RewriteCond %{QUERY_STRING} GLOBALS(=|\[|\%[0-9A-Z]{0,2}) [OR]
# Block out any script trying to modify a _REQUEST variable via URL
RewriteCond %{QUERY_STRING} _REQUEST(=|\[|\%[0-9A-Z]{0,2})
# Return 403 Forbidden header and show the content of the root homepage
RewriteRule .* index.php [F]
#
########## End - Rewrite rules to block out some common exploits


########## Begin - Custom redirects
#
# If you need to redirect some pages, or set a canonical non-www to
# www redirect (or vice versa), place that code here. Ensure those
# redirects use the correct RewriteRule syntax and the [R=301,L] flags.
#
########## End - Custom redirects


# Uncomment following line if your webserver's URL
# is not directly related to physical file paths.
# Update Your Joomla! Directory (just / for root)

# RewriteBase /


########## Begin - Joomla! core SEF Section
#
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
#
# If the requested path and file is not /index.php and the request
# has not already been internally rewritten to the index.php script
RewriteCond %{REQUEST_URI} !^/index\.php
# and the request is for root, or for an extensionless URL, or the
# requested URL ends with one of the listed extensions
RewriteCond %{REQUEST_URI} (/[^.]*|\.(php|html?|feed|pdf|raw))$ [NC]
# and the requested path and file doesn't directly match a physical file
RewriteCond %{REQUEST_FILENAME} !-f
# and the requested path and file doesn't directly match a physical folder
RewriteCond %{REQUEST_FILENAME} !-d
# internally rewrite the request to the index.php script
RewriteRule .* index.php [L]
#
########## End - Joomla! core SEF Section

[edited by: ergophobe at 1:30 am (utc) on Mar 25, 2011]
[edit reason] removed backslashes for Apache 1.3 compatibility as per g1smd [/edit]

g1smd

11:58 am on Mar 1, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Joomla 1.6

##
# @version $Id: BASED ON htaccess.txt 20196 2011-01-09 02:40:25Z ian $
# @MODIFIED 2011-02-26
# @packageJoomla
# @copyrightCopyright (C) 2005 - 2010 Open Source Matters. All rights reserved.
# @licenseGNU General Public License version 2 or later; see LICENSE.txt
##

##
# READ THIS COMPLETELY IF YOU CHOOSE TO USE THIS FILE!
#
# The line just below this section: 'Options +FollowSymLinks' may cause problems
# with some server configurations. It is required for use of mod_rewrite, but may already
# be set by your server administrator in a way that dissallows changing it in
# your .htaccess file. If using it causes your server to error out, comment it out (add # to
# beginning of line), reload your site in your browser and test your sef url's. If they work,
# it has been set by your server administrator and you do not need it set here.
##

## Can be commented out if causes errors, see notes above.
Options +FollowSymLinks

## Mod_rewrite in use.

RewriteEngine On

## Begin - Rewrite rules to block out some common exploits.
# If you experience problems on your site block out the operations listed below
# This attempts to block the most common type of exploit `attempts` to Joomla!
#
# Block out any script trying to base64_encode data within the URL.
RewriteCond %{QUERY_STRING} base64_encode[^(]*\([^)]*\) [OR]
# Block out any script that includes a <script> tag in URL.
RewriteCond %{QUERY_STRING} (<|%3C)([^s]*s)+cript.*(>|%3E) [NC,OR]
# Block out any script trying to set a PHP GLOBALS variable via URL.
RewriteCond %{QUERY_STRING} GLOBALS(=|\[|\%[0-9A-Z]{0,2}) [OR]
# Block out any script trying to modify a _REQUEST variable via URL.
RewriteCond %{QUERY_STRING} _REQUEST(=|\[|\%[0-9A-Z]{0,2})
# Return 403 Forbidden header and show the content of the root homepage
RewriteRule .* index.php [F]
#
## End - Rewrite rules to block out some common exploits.

## Begin - Custom redirects
#
# If you need to redirect some pages, or set a canonical non-www to
# www redirect (or vice versa), place that code here. Ensure those
# redirects use the correct RewriteRule syntax and the [R=301,L] flags.
#
## End - Custom redirects

##
# Uncomment following line if your webserver's URL
# is not directly related to physical file paths.
# Update Your Joomla! Directory (just / for root).
##

# RewriteBase /

## Begin - Joomla! core SEF Section.
#
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
#
# If the requested path and file is not /index.php and the request
# has not already been internally rewritten to the index.php script
RewriteCond %{REQUEST_URI} !^/index\.php
# and the request is for something within the component folder,
# or for the site root, or for an extensionless URL, or the
# requested URL ends with one of the listed extensions
RewriteCond %{REQUEST_URI} /component/|(/[^.]*|\.(php|html?|feed|pdf|raw))$ [NC]
# and the requested path and file doesn't directly match a physical file
RewriteCond %{REQUEST_FILENAME} !-f
# and the requested path and file doesn't directly match a physical folder
RewriteCond %{REQUEST_FILENAME} !-d
# internally rewrite the request to the index.php script
RewriteRule .* index.php [L]
#
## End - Joomla! core SEF Section.

[edited by: ergophobe at 1:30 am (utc) on Mar 25, 2011]
[edit reason] removed backslashes for Apache 1.3 compatibility as per g1smd [/edit]

ergophobe

4:44 pm on Mar 1, 2011 (gmt 0)

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Thanks g1!

I'm wondering... there was some discussion when you and Jim worked through Wordpress (less so for Drupal), but I'm curious. I understand this makes the rewrites way more efficient, but have you benchmarked at all to get a sense of the magnitude of the effect?

Obviously, one thing about this fix is it gets invoked several times per page - images, css, javascript files etc. So small changes per request add up to much larger changes per page.

I'm just trying to get a sense of this compared to, say, optimizing a slow query, which is commonly the worst bottleneck on a Joomla or Drupal site and can slow page generation down by seconds, but commonly by hundreds of milliseconds.

ergophobe

4:56 pm on Mar 1, 2011 (gmt 0)

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



PS - added to CMS FAQ

g1smd

4:59 pm on Mar 1, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Several people have attempted to measure the impact of the changes, but it is difficult because these processes happen before any PHP that could set a timer is loaded.

The more requests per page, and the busier the site, the more this will have an impact in staving off an early server upgrade. Mod_rewrite processing should see at least a factor of three improvement. A page with a lot of images could see even more improvement.

The Joomla changes have taken 5 months to get as far as being "ready to commit", but the trunk is stalled for 1.6.1 at the moment so these changes likely won't see the light of day for several more months.

The WordPress changes were rejected within 20 minutes as a "WontFix". The Drupal changes haven't been finalised, we are waiting for more input to the relevant WebmasterWorld thread before progressing it further.

ergophobe

5:03 pm on Mar 1, 2011 (gmt 0)

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



I'll have to get back to the Drupal changes. That thread was active at a time when I was switching servers and getting all kinds of 500 errors and "404 misdirects" ;-) and generally struggling just to get things running, so I went back the distro.

Now I have the live sites running and some sandbox sites setup, though, I can give it a try now.

In terms of performance, this would have to be tested on a standalone box using something like Apache Bench. I haven't done something like that in a long time though.

g1smd

2:50 pm on Mar 8, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



One minor change to both files.

This code
RewriteCond %{QUERY_STRING} (\<|%3C)([^s]*s)+cript.*(\>|%3E) [NC,OR]


should be:
RewriteCond %{QUERY_STRING} (<|%3C)([^s]*s)+cript.*(>|%3E) [NC,OR]


as reported at: [joomlacode.org...] where original code breaks in Apache 1.3.

[edited by: ergophobe at 9:27 pm (utc) on Mar 24, 2011]
[edit reason] Original posts edited to reflect this change as per g1smd [/edit]

ergophobe

8:55 pm on Mar 8, 2011 (gmt 0)

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



I didn't think < would need escaping, but it surprises me that it creates a problem.

When I'm lazy, I escape anything I'm not sure about in a regex always assuming the it's rarely a problem unless it's something with a special meaning as, say, a letter "d" which "escaped" would be \d... which obviously creates problems (matches any digit, but not a letter d).

Since neither < nor \< have a special meaning (AFAIK), I'm surprised it isn't just parsed normally.

(?<=exp) and (?<!exp) are zero-width positive and negative lookbehinds, but only have a special meaning as part of a longer combo

g1smd

9:16 pm on Mar 8, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Adding unwanted escaping causes Apache 1.3 to throw a wobbler (who knew?), but that problem has been eradicated in Apache 2.x.

It has taken 5 years for anyone to notice the problem within Joomla (well, five years for someone to report it).

So, only add the escaping to things that actually need it.

ergophobe

8:57 pm on Mar 9, 2011 (gmt 0)

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Usually for rewrites, I am more careful, but I probably do 100 regex searches on many days in the course of my work, and I churn those out fast b/c there is no consequence for them being wrong (searching for needles in haystacks, and I'm not altering the haystack or the needle).

And on other topics, Apache 1.3? Is that the thing I used to use to run my Wordpress 2.3 site ;-)

spadilla

1:30 am on Mar 19, 2011 (gmt 0)

5+ Year Member



Thank you g1! I have been pulling my hair out dealing with htaccess issues and Joomla 1.5!

I had been using nikosdion's master htaccess in the past: [snipt.net...] - Interested to know your thoughts on this file.

Thanks again!

g1smd

1:49 am on Mar 19, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



The file has multiple logic, coding and syntax errors. Several of the rules can never work as originally coded as they will never match any request. Many of the rules are highly inefficient and could almost stall a server with moderate load. Some rules need extra modules loaded or will only work on Apache 2.x, but those facts are not noted.

However, all is not lost. The file has been edited in recent days.
The complete list of changes: [docs.joomla.org...]
The list of comments for each change: [docs.joomla.org...]

spadilla

2:26 am on Mar 22, 2011 (gmt 0)

5+ Year Member



Just tested out your htaccess on a large site with the SH404 component enabled and I am seeing a noticeable speed increase. Thanks again for your hard work!

g1smd

7:58 am on Mar 22, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Could you sign up to the Joomlacode site and make a note in the issue tracker (links to 1.5 and 1.6 in first post) there that you have tested it?

spadilla

8:32 pm on Mar 23, 2011 (gmt 0)

5+ Year Member



Done! And thanks again!

g1smd

8:41 pm on Mar 23, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Thanks! I'm hoping if many people report that it works, they'll actually use it for real.

g1smd

12:02 am on Mar 26, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



The Joomla 1.6 .htaccess file has been updated to include the code at the top of this thread.

[joomlacode.org...]
[joomlacode.org...]

g1smd

9:04 pm on Mar 26, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



@spadilla There's some issues with the 2.3 "master file". I have listed the proposed 2.4 changes and the reasons for those changes at: [codereview.appspot.com...] Be sure to click on "expand comments (e)".

The differences are also listed at: [snipt.net...] and the complete new file can be found at: [snipt.net...] and [code.google.com...] and [docs.joomla.org...]

g1smd

8:26 pm on Mar 27, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Still looking for testers to check out the version 1.5 code at [joomlacode.org...] and make a note about it on that tracker.

g1smd

10:35 pm on Apr 3, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



The joomla 1.5 .htaccess file has been updated to include the code at the top of this thread.

[joomlacode.org...]
[joomlacode.org...]

ergophobe

8:31 pm on Apr 4, 2011 (gmt 0)

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Good work!

spadilla

6:39 pm on Apr 5, 2011 (gmt 0)

5+ Year Member



Awesome! Glad to see they added your htaccess to patch 1.5.23 today!
 

Featured Threads

Hot Threads This Week

Hot Threads This Month