Welcome to WebmasterWorld Guest from 54.166.5.230

Forum Moderators: ergophobe

Message Too Old, No Replies

Replacement .htaccess files for Joomla 1.5 and Joomla 1.6

Large speed boost gained by optimising the mod_rewrite code.

     
11:48 am on Mar 1, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


The default
.htaccess
file supplied with Joomla 1.5 and Joomla 1.6 has a number of deficiencies and inefficiencies.

Below, replacement files, to fix these issues.

See also:
[joomlacode.org...] (Joomla 1.5)
[joomlacode.org...] (Joomla 1.6)
11:51 am on Mar 1, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


Joomla 1.5

##
# @version $Id: BASED ON htaccess.txt 14401 2010-01-26 14:10:00Z louis $
# @MODIFIED 2011-02-25
# @package Joomla
# @copyright Copyright (C) 2005 - 2010 Open Source Matters. All rights reserved.
# @license http://www.gnu.org/copyleft/gpl.html GNU/GPL
# Joomla! is Free Software
##


#####################################################
# READ THIS COMPLETELY IF YOU CHOOSE TO USE THIS FILE
#
# The line just below this section: 'Options +FollowSymLinks' may cause problems
# with some server configurations. It is required for use of mod_rewrite, but may already
# be set by your server administrator in a way that dissallows changing it in
# your .htaccess file. If using it causes your server to error out, comment it out (add # to
# beginning of line), reload your site in your browser and test your sef url's. If they work,
# it has been set by your server administrator and you do not need it set here.
#
#####################################################

## Can be commented out if causes errors, see notes above.
Options +FollowSymLinks

#
# mod_rewrite in use

RewriteEngine On

########## Begin - Rewrite rules to block out some common exploits
## If you experience problems on your site block out the operations listed below
## This attempts to block the most common type of exploit `attempts` to Joomla!
#
## Deny access to extension xml files (uncomment out to activate)
#<Files ~ "\.xml$">
#Order allow,deny
#Deny from all
#Satisfy all
#</Files>
## End of deny access to extension xml files
# Block out any script trying to set a mosConfig value through the URL
RewriteCond %{QUERY_STRING} mosConfig_[a-zA-Z_]{1,21}(=|\%3D) [OR]
# Block out any script trying to base64_encode data within the URL
RewriteCond %{QUERY_STRING} base64_encode[^(]*\([^)]*\) [OR]
# Block out any script that includes a <script> tag in URL
RewriteCond %{QUERY_STRING} (<|%3C)([^s]*s)+cript.*(>|%3E) [NC,OR]
# Block out any script trying to set a PHP GLOBALS variable via URL
RewriteCond %{QUERY_STRING} GLOBALS(=|\[|\%[0-9A-Z]{0,2}) [OR]
# Block out any script trying to modify a _REQUEST variable via URL
RewriteCond %{QUERY_STRING} _REQUEST(=|\[|\%[0-9A-Z]{0,2})
# Return 403 Forbidden header and show the content of the root homepage
RewriteRule .* index.php [F]
#
########## End - Rewrite rules to block out some common exploits


########## Begin - Custom redirects
#
# If you need to redirect some pages, or set a canonical non-www to
# www redirect (or vice versa), place that code here. Ensure those
# redirects use the correct RewriteRule syntax and the [R=301,L] flags.
#
########## End - Custom redirects


# Uncomment following line if your webserver's URL
# is not directly related to physical file paths.
# Update Your Joomla! Directory (just / for root)

# RewriteBase /


########## Begin - Joomla! core SEF Section
#
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
#
# If the requested path and file is not /index.php and the request
# has not already been internally rewritten to the index.php script
RewriteCond %{REQUEST_URI} !^/index\.php
# and the request is for root, or for an extensionless URL, or the
# requested URL ends with one of the listed extensions
RewriteCond %{REQUEST_URI} (/[^.]*|\.(php|html?|feed|pdf|raw))$ [NC]
# and the requested path and file doesn't directly match a physical file
RewriteCond %{REQUEST_FILENAME} !-f
# and the requested path and file doesn't directly match a physical folder
RewriteCond %{REQUEST_FILENAME} !-d
# internally rewrite the request to the index.php script
RewriteRule .* index.php [L]
#
########## End - Joomla! core SEF Section

[edited by: ergophobe at 1:30 am (utc) on Mar 25, 2011]
[edit reason] removed backslashes for Apache 1.3 compatibility as per g1smd [/edit]

11:58 am on Mar 1, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


Joomla 1.6

##
# @version $Id: BASED ON htaccess.txt 20196 2011-01-09 02:40:25Z ian $
# @MODIFIED 2011-02-26
# @packageJoomla
# @copyrightCopyright (C) 2005 - 2010 Open Source Matters. All rights reserved.
# @licenseGNU General Public License version 2 or later; see LICENSE.txt
##

##
# READ THIS COMPLETELY IF YOU CHOOSE TO USE THIS FILE!
#
# The line just below this section: 'Options +FollowSymLinks' may cause problems
# with some server configurations. It is required for use of mod_rewrite, but may already
# be set by your server administrator in a way that dissallows changing it in
# your .htaccess file. If using it causes your server to error out, comment it out (add # to
# beginning of line), reload your site in your browser and test your sef url's. If they work,
# it has been set by your server administrator and you do not need it set here.
##

## Can be commented out if causes errors, see notes above.
Options +FollowSymLinks

## Mod_rewrite in use.

RewriteEngine On

## Begin - Rewrite rules to block out some common exploits.
# If you experience problems on your site block out the operations listed below
# This attempts to block the most common type of exploit `attempts` to Joomla!
#
# Block out any script trying to base64_encode data within the URL.
RewriteCond %{QUERY_STRING} base64_encode[^(]*\([^)]*\) [OR]
# Block out any script that includes a <script> tag in URL.
RewriteCond %{QUERY_STRING} (<|%3C)([^s]*s)+cript.*(>|%3E) [NC,OR]
# Block out any script trying to set a PHP GLOBALS variable via URL.
RewriteCond %{QUERY_STRING} GLOBALS(=|\[|\%[0-9A-Z]{0,2}) [OR]
# Block out any script trying to modify a _REQUEST variable via URL.
RewriteCond %{QUERY_STRING} _REQUEST(=|\[|\%[0-9A-Z]{0,2})
# Return 403 Forbidden header and show the content of the root homepage
RewriteRule .* index.php [F]
#
## End - Rewrite rules to block out some common exploits.

## Begin - Custom redirects
#
# If you need to redirect some pages, or set a canonical non-www to
# www redirect (or vice versa), place that code here. Ensure those
# redirects use the correct RewriteRule syntax and the [R=301,L] flags.
#
## End - Custom redirects

##
# Uncomment following line if your webserver's URL
# is not directly related to physical file paths.
# Update Your Joomla! Directory (just / for root).
##

# RewriteBase /

## Begin - Joomla! core SEF Section.
#
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
#
# If the requested path and file is not /index.php and the request
# has not already been internally rewritten to the index.php script
RewriteCond %{REQUEST_URI} !^/index\.php
# and the request is for something within the component folder,
# or for the site root, or for an extensionless URL, or the
# requested URL ends with one of the listed extensions
RewriteCond %{REQUEST_URI} /component/|(/[^.]*|\.(php|html?|feed|pdf|raw))$ [NC]
# and the requested path and file doesn't directly match a physical file
RewriteCond %{REQUEST_FILENAME} !-f
# and the requested path and file doesn't directly match a physical folder
RewriteCond %{REQUEST_FILENAME} !-d
# internally rewrite the request to the index.php script
RewriteRule .* index.php [L]
#
## End - Joomla! core SEF Section.

[edited by: ergophobe at 1:30 am (utc) on Mar 25, 2011]
[edit reason] removed backslashes for Apache 1.3 compatibility as per g1smd [/edit]

4:44 pm on Mar 1, 2011 (gmt 0)

Moderator This Forum

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 25, 2002
posts:8418
votes: 201


Thanks g1!

I'm wondering... there was some discussion when you and Jim worked through Wordpress (less so for Drupal), but I'm curious. I understand this makes the rewrites way more efficient, but have you benchmarked at all to get a sense of the magnitude of the effect?

Obviously, one thing about this fix is it gets invoked several times per page - images, css, javascript files etc. So small changes per request add up to much larger changes per page.

I'm just trying to get a sense of this compared to, say, optimizing a slow query, which is commonly the worst bottleneck on a Joomla or Drupal site and can slow page generation down by seconds, but commonly by hundreds of milliseconds.
4:56 pm on Mar 1, 2011 (gmt 0)

Moderator This Forum

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 25, 2002
posts:8418
votes: 201


PS - added to CMS FAQ
4:59 pm on Mar 1, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


Several people have attempted to measure the impact of the changes, but it is difficult because these processes happen before any PHP that could set a timer is loaded.

The more requests per page, and the busier the site, the more this will have an impact in staving off an early server upgrade. Mod_rewrite processing should see at least a factor of three improvement. A page with a lot of images could see even more improvement.

The Joomla changes have taken 5 months to get as far as being "ready to commit", but the trunk is stalled for 1.6.1 at the moment so these changes likely won't see the light of day for several more months.

The WordPress changes were rejected within 20 minutes as a "WontFix". The Drupal changes haven't been finalised, we are waiting for more input to the relevant WebmasterWorld thread before progressing it further.
5:03 pm on Mar 1, 2011 (gmt 0)

Moderator This Forum

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 25, 2002
posts:8418
votes: 201


I'll have to get back to the Drupal changes. That thread was active at a time when I was switching servers and getting all kinds of 500 errors and "404 misdirects" ;-) and generally struggling just to get things running, so I went back the distro.

Now I have the live sites running and some sandbox sites setup, though, I can give it a try now.

In terms of performance, this would have to be tested on a standalone box using something like Apache Bench. I haven't done something like that in a long time though.
2:50 pm on Mar 8, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


One minor change to both files.

This code
RewriteCond %{QUERY_STRING} (\<|%3C)([^s]*s)+cript.*(\>|%3E) [NC,OR]


should be:
RewriteCond %{QUERY_STRING} (<|%3C)([^s]*s)+cript.*(>|%3E) [NC,OR]


as reported at: [joomlacode.org...] where original code breaks in Apache 1.3.

[edited by: ergophobe at 9:27 pm (utc) on Mar 24, 2011]
[edit reason] Original posts edited to reflect this change as per g1smd [/edit]

8:55 pm on Mar 8, 2011 (gmt 0)

Moderator This Forum

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 25, 2002
posts:8418
votes: 201


I didn't think < would need escaping, but it surprises me that it creates a problem.

When I'm lazy, I escape anything I'm not sure about in a regex always assuming the it's rarely a problem unless it's something with a special meaning as, say, a letter "d" which "escaped" would be \d... which obviously creates problems (matches any digit, but not a letter d).

Since neither < nor \< have a special meaning (AFAIK), I'm surprised it isn't just parsed normally.

(?<=exp) and (?<!exp) are zero-width positive and negative lookbehinds, but only have a special meaning as part of a longer combo
9:16 pm on Mar 8, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


Adding unwanted escaping causes Apache 1.3 to throw a wobbler (who knew?), but that problem has been eradicated in Apache 2.x.

It has taken 5 years for anyone to notice the problem within Joomla (well, five years for someone to report it).

So, only add the escaping to things that actually need it.
8:57 pm on Mar 9, 2011 (gmt 0)

Moderator This Forum

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 25, 2002
posts:8418
votes: 201


Usually for rewrites, I am more careful, but I probably do 100 regex searches on many days in the course of my work, and I churn those out fast b/c there is no consequence for them being wrong (searching for needles in haystacks, and I'm not altering the haystack or the needle).

And on other topics, Apache 1.3? Is that the thing I used to use to run my Wordpress 2.3 site ;-)
1:30 am on Mar 19, 2011 (gmt 0)

Junior Member

5+ Year Member

joined:Aug 18, 2008
posts:115
votes: 0


Thank you g1! I have been pulling my hair out dealing with htaccess issues and Joomla 1.5!

I had been using nikosdion's master htaccess in the past: [snipt.net...] - Interested to know your thoughts on this file.

Thanks again!
1:49 am on Mar 19, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


The file has multiple logic, coding and syntax errors. Several of the rules can never work as originally coded as they will never match any request. Many of the rules are highly inefficient and could almost stall a server with moderate load. Some rules need extra modules loaded or will only work on Apache 2.x, but those facts are not noted.

However, all is not lost. The file has been edited in recent days.
The complete list of changes: [docs.joomla.org...]
The list of comments for each change: [docs.joomla.org...]
2:26 am on Mar 22, 2011 (gmt 0)

Junior Member

5+ Year Member

joined:Aug 18, 2008
posts:115
votes: 0


Just tested out your htaccess on a large site with the SH404 component enabled and I am seeing a noticeable speed increase. Thanks again for your hard work!
7:58 am on Mar 22, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


Could you sign up to the Joomlacode site and make a note in the issue tracker (links to 1.5 and 1.6 in first post) there that you have tested it?
8:32 pm on Mar 23, 2011 (gmt 0)

Junior Member

5+ Year Member

joined:Aug 18, 2008
posts:115
votes: 0


Done! And thanks again!
8:41 pm on Mar 23, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


Thanks! I'm hoping if many people report that it works, they'll actually use it for real.
12:02 am on Mar 26, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


The Joomla 1.6 .htaccess file has been updated to include the code at the top of this thread.

[joomlacode.org...]
[joomlacode.org...]
9:04 pm on Mar 26, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


@spadilla There's some issues with the 2.3 "master file". I have listed the proposed 2.4 changes and the reasons for those changes at: [codereview.appspot.com...] Be sure to click on "expand comments (e)".

The differences are also listed at: [snipt.net...] and the complete new file can be found at: [snipt.net...] and [code.google.com...] and [docs.joomla.org...]
8:26 pm on Mar 27, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


Still looking for testers to check out the version 1.5 code at [joomlacode.org...] and make a note about it on that tracker.
10:35 pm on Apr 3, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


The joomla 1.5 .htaccess file has been updated to include the code at the top of this thread.

[joomlacode.org...]
[joomlacode.org...]
8:31 pm on Apr 4, 2011 (gmt 0)

Moderator This Forum

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 25, 2002
posts:8418
votes: 201


Good work!
6:39 pm on Apr 5, 2011 (gmt 0)

Junior Member

5+ Year Member

joined:Aug 18, 2008
posts:115
votes: 0


Awesome! Glad to see they added your htaccess to patch 1.5.23 today!
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members