homepage Welcome to WebmasterWorld Guest from 54.205.241.107
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Code, Content, and Presentation / Apache Web Server
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL & phranque

Apache Web Server Forum

    
How big is too big for an htaccess file?
I have a huge site with 100,000's of pages. htaccess file is over 25 megs.
acimag

5+ Year Member



 
Msg#: 4536845 posted 2:57 pm on Jan 17, 2013 (gmt 0)

I know that is not good at all. There are patterns that I might be able to do lets say 25,000 lines at a time but it involves seeing the pattern, matching and then converting to lower case. is that possible. I'll show you 2 lines maybe you can see the pattern and help... I hope.

RewriteRule ^Emergency-Fast-Business-Rental/Hot/Connecticut/Fairfield/Aqua_Vista_Fast_Temp_Quick_Gas_Heaters_Rentals.html http://www.example.com/aqua_vista-ct-temporary_blah_blah_hot_heaters.html


Now this pattern changes the beginning before connecticuit and then after 'Aqua_Vista' and even the state and county change for every listing. I don't want a bad site performance.

 

g1smd

WebmasterWorld Senior Member g1smd us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4536845 posted 3:16 pm on Jan 17, 2013 (gmt 0)

If possible the new URL should use hyphens, not underscores.

This then becomes a simple problem to rewrite (that's rewrite not redirect) all requests with underscores to a PHP script that manipulates the URL and sends the HTTP 301 header and the new URL header OR sends 404 Not Found when there is no new URL to redirect to.

This means the htaccess performance for all requests is quick as the file is short.

The PHP file can be as large or as small as you like, it will only be invoked for requests that need to be redirected.

The PHP file could be as simple as old and new URLs as pairs in an array, or patterns covering multiple URLs in an array, or other logic as required. Whatever you do, Regular Expressions will probably feature quite highly.

acimag

5+ Year Member



 
Msg#: 4536845 posted 4:38 pm on Jan 17, 2013 (gmt 0)

So you telling me my htaccess file can be as large as I want like 50 megs and it won't slow down the site?

Thats insane. I thought the file would atleast have to load so all the rewrites would work.

as for everything else you said its way over my head and i've tried reading a lot about this subject but found very limited resources.

g1smd

WebmasterWorld Senior Member g1smd us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4536845 posted 4:40 pm on Jan 17, 2013 (gmt 0)

I said the PHP file can be as large as you like.

If you rewrite underscore requests to a PHP script, then that's one line of code in the htaccess file.

RewriteRule _ /handlethis.php [L]

The PHP file does the real work, looking up the new URL that replaces the old, and sending the correct HTTP headers (301 or 404) back to the browser.

You can have thousands of lines in htaccess if you want, but the code becomes unmanageable. There are better solutions.

acimag

5+ Year Member



 
Msg#: 4536845 posted 5:35 pm on Jan 17, 2013 (gmt 0)

yeah i just finished putting in all the redirects the file is now 73mgs.

and the site runs VERY slow now. I noticed i do not have the [L] at the end. does that matter.

acimag

5+ Year Member



 
Msg#: 4536845 posted 5:48 pm on Jan 17, 2013 (gmt 0)

correction I just added the [L] and is VERY SLOW. any idea's?

topr8

WebmasterWorld Senior Member topr8 us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4536845 posted 6:31 pm on Jan 17, 2013 (gmt 0)

>>correction I just added the [L] and is VERY SLOW. any idea's?

of course, 50 megs is way too big for an htaccess file
and you could also have rules in there that are taking a long time (relatively) to process.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4536845 posted 10:48 pm on Jan 17, 2013 (gmt 0)

TWENTY-FIVE MEGABYTES?

You did say megs, right, not kilobytes?

That makes this entire thread rhetorical. 25 MB is too big for anything,* let alone an htaccess file.

In particular, keep in mind that the full htaccess is read at every single request, not just pages. Even the [L] flag just means "Don't execute any more mod_rewrite rules this time around", not "Stop reading this file".


* That is: Anything the user opens without advance notice. If you've linked to a 600MB PDF, human decency requires you to say so.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4536845 posted 10:22 pm on Feb 9, 2013 (gmt 0)

:: returning after reading first post more slowly ::

There are patterns that I might be able to do lets say 25,000 lines at a time but it involves seeing the pattern, matching and then converting to lower case. is that possible. I'll show you 2 lines maybe you can see the pattern and help.


Unfortunately you didn't show two lines. You only ended up showing one line, and you can't deduce a pattern from that.

:: vague mental association with some process in mathematics or possibly historical linguistics where you do in fact make a rule based on only one instance ::

mod_rewrite alone can't change the case of an URL. It can detect requests in a case-insensitive way, like picking up .jpg, .Jpg and .JPG in a single rule. But there's no straightforward* way to make them all output .jpg other than by individual redirects. The same applies to any mass replacement like lowline-to-hyphen (or, ahem, vice versa**).

That's why g1 originally said "php script". (If you read the apache subforum regularly, you will see that g1 very often says "php script". Probably because people only come here when they've got questions that would defeat the ordinary htaccess. Nothing to do with the fact that any solution involving php bumps the follow-up questions into a different subforum ;)) Among other things, php can change cases standing on its head. Heck, even javascript can do it. But mod_rewrite can't.


* There's a post by jdmorgan illustrating the non-straightforward way. It relies on flags like [C] and [N] that normal people approach with EXTREME CAUTION.

** Yes, I know, it's just me. But I always think hyphens look spammy. get-your-cheap-widgets-here dot biz, kind of thing.

Hoople

10+ Year Member



 
Msg#: 4536845 posted 6:14 am on Feb 10, 2013 (gmt 0)

This sounds like a classic case of trying to fix the effect and not the cause(s).

How long a history of change as well as how many big issues you are trying to hit with one brushstroke?

Are there multiple site redesign URL changes that are you covering and their time period?

londrum

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 4536845 posted 2:15 pm on Feb 10, 2013 (gmt 0)

have you just got one htaccess file in the root?

maybe you could split the file up into lots of smaller files, and put each one into the directory that it affects.

that way you wont have to read the entire thing every single time.

acimag

5+ Year Member



 
Msg#: 4536845 posted 7:09 pm on Feb 12, 2013 (gmt 0)

I get it. . Thanks Lucy. I see my problem.

Hoop. Its just a massive site we are reduing and the URL's and wanted not to lose search ranking.

Londrum. I think your onto something. Thanks!

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved