Welcome to WebmasterWorld Guest from 34.236.216.93

Forum Moderators: Ocean10000 & phranque

Message Too Old, No Replies

Validating an htaccess file

Is there a way?

     
8:07 am on Sep 1, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:May 16, 2003
posts:109
votes: 0


I have a rather lengthy .htaccess file that seems to be allowing some spiders that I am "rewriting" to another location.

Is there a way to validate my file to insure it is properly structured? Thanks.

5:29 pm on Sept 2, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 21, 1999
posts:2141
votes: 0


Not that I've ever found! Here's [httpd.apache.org] a list of Apache directives, most of them can be used in .htaccess, and each directive has many options. Creating a validator for all of these would very likely be on the same scale as writing the Apache server code I suspect.
7:40 pm on Sept 2, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Sept 28, 2002
posts:505
votes: 0


Hi Guillermo500,

... about a year ago, this thread said "no":
[webmasterworld.com...]

You could first try it on a test server before deploying it onto a production system.

Regards,
R.

11:11 pm on Sept 2, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
posts:25430
votes: 0


Several tricks I use:

1) Break the file into logical sections and test them one-at-a-time.
2) Test the code on a test server, or even in a test subdirectory on a live server.
3) Use WannaBrowser to check user-agent blocking.
4) If you have a problem with some bad guys getting through, scan through the list of blocked user-agents and/or IP addresses, and make sure you have [OR] flags where you want them, and none where you don't want them.
5) Make sure your have "{", "}", "(", and ")" in the correct places; using "()" around server variables can lead to a silent failure. Example:

RewriteCond %(HTTP_USER_AGENT) ^Larbin [NC]

will fail.

Jim

3:34 am on Sept 3, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:May 16, 2003
posts:109
votes: 0


Thanks for the great ideas.

I think my issues can be cleared-up with a few questions if you'll indulge me.

This is one section that troubles me. Is there a problem with the extra "deny from" statements below?

<Limit GET POST PUT>
order allow,deny
allow from all
deny from 24.4.253.122 24.4.253.123 209.26.224.45 towtimes 24.183.144 216.34.244 24.73.161.18 www.protow.com mail.protow.com 24.73.204.166 proxy-server.southeast.rr.com 216.160.95.136 idslppp136.sttl.uswest.net static-64-65-139-27.dsl.sea.eschelon.com transport-development-group.london.cw.net primeinc.com 208.242.194.183 80.58.52.170 64.179.23.178 66.73.48.200 63.64.41.58 66.73.48.200 207.31.249.220 12.151.162.21 63.148.99.247 209.149.201.5 Allresearch.com

# Npbot
deny from 12.148.209.192/26
# cyveillance.com
deny from 63.148.99.224/27
deny from 65.118.41.192/27
# branddimensions.com user-agent: BDFetch
deny from 204.92.59.0/24
# www.markwatch.com user-agent: markwatch
deny from 204.62.224.0/22
deny from 204.62.228.0/23
deny from 206.190.160.0/19
# rocketinfo.com
deny from 209.167.132.224/28
</Limit>

4:19 am on Sept 3, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
posts:25430
votes: 0


guillermo5000,

No, you can have as many Allows and Denys as you like.

You might actually want to add a few more Denys to shorten the list in your first Deny and make it easer to read by grouping IPs and Remote-Hosts separately.

Note that you can block both www.protow.com and mail.protow.com by using Deny from protow.com
However, this would also block xyz.protow.com, so you may not want to do that.

If *none* of these Denys work, try removing the <Limit GET POST PUT> and </Limit> and surrounding the code with <Files *> and </Files> instead.

Jim

4:48 am on Sept 3, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:May 16, 2003
posts:109
votes: 0


Thank you JP. I did find a missing [OR] in my Rewrite statements. And after checking them with WannaBrowser (great tool, thank you), they all are working.

My last questions are rather simple. Can these statements (below) go anywhere in the htaccess file? They currently reside after my "rewrite engine on" command and before my Rewrite statements.

RedirectMatch 301 /cgi-bin/ikonboard/* /cgi-bin/ib3/ikonboard.cgi$1
RedirectMatch 301 /toc.htm$ /toc2.htm$1
RedirectMatch 301 /links.htm$ /resources.htm$1

And lastly; Is the syntax of this line correct?

RewriteRule ^.*$ [google.com...] [R,L]

Thank you all for the help!

4:51 am on Sept 3, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:May 16, 2003
posts:109
votes: 0


Oops, One more tiny question.

I have both of these statements in my htaccess:
Options Includes
Options All

Are they redundant?

Thanks!

5:56 am on Sept 3, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
posts:25430
votes: 0


Can these statements (below) go anywhere in the htaccess file? They currently reside after my "rewrite engine on" command and before my Rewrite statements.

RedirectMatch 301 /cgi-bin/ikonboard/* /cgi-bin/ib3/ikonboard.cgi$1
RedirectMatch 301 /toc.htm$ /toc2.htm$1
RedirectMatch 301 /links.htm$ /resources.htm$1


These directives apply to mod_alias, not to mod_rewrite. Just as a matter of "style" you might want to put them either before RewriteEngine on, or after your last RewriteRule.

Is the syntax of this line correct?

RewriteRule ^.*$ [google.com...] [R,L]

I'd suggest specifying either R=301 or R=302, just to make it obvious whether a permanent or temporary redirect is being done (The default is 302-Temporarily Moved).
You can shorten the pattern to just ".*" (or even to just ".") if you like:


RewriteRule .* http://www.google.com [R=301,L]

I have both of these statements in my htaccess:
Options Includes
Options All

Yes, "All" includes "Includes", so the latter is redundant.
Note that for security purposes, it's a good idea to include "Options -Indexes" after "Options All" unless you wish users to see directory listings of your subdirectories.

See Apache core (for Options), mod_alias, and mod_rewrite at [httpd.apache.org...] for details.

Jim

6:34 am on Sept 3, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:May 16, 2003
posts:109
votes: 0


Thank you Jim. You've been tremendously helpful.

Do you know where I can read about these "Options" commands? I have removed the "options all" until I know more.

6:37 am on Sept 3, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:May 16, 2003
posts:109
votes: 0


I found it, Thanks Jim.

[httpd.apache.org...]

6:45 am on Sept 3, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:May 16, 2003
posts:109
votes: 0


When I remove the "options all" and leave "options includes", my site becomes unavailable.

Why is that?

Thanks.

7:04 am on Sept 3, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
posts:25430
votes: 0


guillermo5000,

You'll need Options FollowSymLinks Includes to use mod_rewrite and SSI, and Options ExecCGI FollowSymLinks Includes if you use any scripts in addition to mod_rewrite and SSI.

Jim