htaccess protection

which site do I put in?

[camille]

Hello,

I have several rewrite conditions:

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^http://mystore.com/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://mystore.com$ [NC]
RewriteCond %{HTTP_REFERER} !^http://mystoreshop.com/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://mystoreshop.com$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.mystore.com/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.mystore.com$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.mystoreshop.com/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.mystoreshop.com$ [NC]
RewriteCond %{HTTP_REFERER} !^https://www.mystoreshop.com/.*$ [NC]
RewriteCond %{HTTP_REFEER} !^https://www.mystoreshop.com$ [NC]
RewriteRule .*\.(jpg¦jpeg¦gif¦png¦bmp)$ - [F,NC]

Options +FollowSymLinks
RewriteEngine on
#
# Canonicalize the domain name: Redirect if the requested
# hostname is non-blank and is NOT precisely "www.mystoreshop.com"
RewriteCond %{HTTP_HOST} .
RewriteCond %{HTTP_HOST} !^www\.mystoreshop\.com$
RewriteRule (.*) [mystoreshop.com...] [R=301,L]
#
# Internally rewrite root directory requests to /store
RewriteCond $1 !^store/
RewriteRule (.*) store/$1 [L]

Does it matter which site I put for "YOURSITE.COM" below? Is there a better way to protect your site then using the example below.

For example:

RewriteCond %{HTTP_USER_AGENT} ^Zeus

RewriteRule ^.* - [F,L]

RewriteCond %{HTTP_REFERER} ^http://www.YOURSITE.COM$

RewriteRule !^http://[^/.]\.YOURSITE.COM.* - [F,L]

This stuff confuses me I am embarrased to say. Camille

[g1smd]

You only need this lot once, right at the top:

Options +FollowSymLinks 
RewriteEngine on

No need to duplicate it further down the page.

.

The code for the ten RewriteCond lines can be considerably simplified, to just two lines, I think:

RewriteCond %{HTTP_REFERER} !^http://(www\.)?mystore(shop)?\.com/.* [NC] 
RewriteCond %{HTTP_REFERER} !^https://www.mystoreshop.com/.* [NC]
RewriteRule .*\.(jpe?g¦gif¦png¦bmp)$ - [F,NC]

Note the simplification in the file extensions list too.

.

Are you sure you want *all* to be redirected?

# Internally rewrite root directory requests to /store 
RewriteCond $1 !^store/ 
RewriteRule (.*) store/$1 [L]

What about /robots.txt or your Google/Yahoo/Live WMT user-ID verification file?

Should the $1 be %1 on the first line ... like this?

# Internally rewrite root directory requests to /store 
RewriteCond [b]%1[/b] !^store/ 
RewriteRule (.*) store/$1 [L]

Actually, that rule might be completely wrong. I might be completely wrong.

.

This line is incorrect:

RewriteRule !^http://[^/.]\.YOURSITE.COM.* - [F,L]

RewriteRule sees the only the folder and filepath, not the host name.

Do you need another RewriteCond to test %{HTTP_HOST) there?

Additionally, [F] always implies [L] so you can just use [F] and it will work just fine.

Fix up your code as far as you can, then repost it here.

A couple of years ago this stuff was complete rocket science to me, so I do feel your pain.

[jdMorgan]

Further simplification - There is never any need to prefix or append ".*" to an un-anchored pattern:

RewriteCond %{HTTP_REFERER} !^http://(www\.)?mystore(shop)?\.com [NC]
RewriteCond %{HTTP_REFERER} !^https://www\.mystoreshop\.com [NC]
RewriteRule \.(jpe?g¦gif¦png¦bmp)$ - [F,NC]

Replace the broken pipe "¦" characters in the rule pattern with solid pipes before use; Posting on this forum modifies the pipe characters.

Syntactically, the following rule is correct: You *do* want to use the $1 back-reference to the matched RewriteRule pattern, and not a %n back-reference to the preceding RewriteCond matched pattern (in this case, there is none, so it would be undefined):

# Internally rewrite root directory requests to /store
RewriteCond $1 !^store/
RewriteRule (.*) store/$1 [L]

However, the question about "Do you really want to match all? -- Including standard robots.txt, Sitemaps, Webmaster tools auth keys, privacy-policy and site-content-rating files still stands. Adding exceptions (additional negative-match RewriteConds) for these files (if you use them or may use them in the future) is recommended.

Jim

[edited by: jdMorgan at 4:57 pm (utc) on Sep. 26, 2008]

[camille]

Ok, I need to print this out and analyze. Ill be back later. Thanks for your help. Camille

[jdMorgan]

Because you force an external redirect to canonicalize the domain name, your 'security' rules need only consider refererrals from the canonical domain(s).

Noting that you provided for HTTPS in the referrer-based rule, I would add that if you do use HTTPS, then your domain canonicalization rule will need to be duplicated (or modified), so that both http and https domains are canonicalized without changing the protocol from https to http, or vice-versa.

The simplest approach is:

# Canonicalize the domain name: Redirect if the http-requested
# hostname is non-blank and is NOT precisely "www.mystoreshop.com"
RewriteCond %{SERVER_PORT} [b]!^4[/b]43$
RewriteCond %{HTTP_HOST} .
RewriteCond %{HTTP_HOST} !^www\.mystoreshop\.com$
RewriteRule (.*) http://www.mystoreshop.com/$1 [R=301,L]
#
# Canonicalize the domain name: Redirect if the htt[b]ps[/b]-requested
# hostname is non-blank and is NOT precisely "www.mystoreshop.com"
RewriteCond %{SERVER_PORT} ^443$
RewriteCond %{HTTP_HOST} .
RewriteCond %{HTTP_HOST} !^www\.mystoreshop\.com$
RewriteRule (.*) htt[b]ps:[/b]//www.mystoreshop.com/$1 [R=301,L]

Jim

[edited by: jdMorgan at 5:45 pm (utc) on Sep. 26, 2008]

[camille]

Hi thanks again for the help. I have this:

Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_REFERER} !^http://(www\.)?mystore(shop)?\.com [NC]
RewriteCond %{HTTP_REFERER} !^https://www\.mystoreshop\.com [NC]
RewriteRule .*\.(jpg¦jpeg¦gif¦png¦bmp)$ - [F,NC]
# Canonicalize the domain name: Redirect if the http-requested
# hostname is non-blank and is NOT precisely "www.mystoreshop.com"
RewriteCond %{SERVER_PORT} !^443$
RewriteCond %{HTTP_HOST} .
RewriteCond %{HTTP_HOST} !^www\.mystoreshop\.com$
RewriteRule (.*) [mystoreshop.com...] [R=301,L]
#
# Canonicalize the domain name: Redirect if the https-requested
# hostname is non-blank and is NOT precisely "www.mystoreshop.com"
RewriteCond %{SERVER_PORT} ^443$
RewriteCond %{HTTP_HOST} .
RewriteCond %{HTTP_HOST} !^www\.mystoreshop\.com$
RewriteRule (.*) [mystoreshop.com...] [R=301,L]

# Internally rewrite root directory requests to /store
RewriteCond $1 !^store/
RewriteRule (.*) store/$1 [L]

You mention that it is recommended to write exceptions. I'll have to read about this. What is the draw-back to not having exceptions?

Im still working on the protection part..

[jdMorgan]

You will determine whether or not you have any URLs which you do not want to be rewritten. Examples might include URLs leading to your admin and stats directories, and/or "well-known" subdirectories such as those used for privacy-policy and content-labeling declarations, and files such as robots.txt, sitemap, and webmaster tool account authorization files, which are conventionally *not* handled by scripts and are installed in the Web root-level directory.

Basically, it is quite uncommon to want to rewrite *absolutely all* URL requests to a script.

Jim

[camille]

Ok, sounds complicated! Ill read into how to do my rewrites.

Thanks Jim.

Camille

[jdMorgan]

.htaccess is a server configuration file so it is, by necessity, quite complicated.

The search engine forums often discuss what you may need to do to prevent ranking problems -- such things as protocol, domain, URL, and query canonicalization. In general, we discuss *how* to implement these things here, and not so much of the "why."

I'd imagine you could spend six months reading posts here at WebmasterWorld and studying the underlying server documentation. And that would be a fairly good start... ;) The more you know and learn now, the fewer problems you might have to address in the future; We get posts all the time saying things like, "I changed all of my URLs, and my site has not been listed in the search engines for eith months now!" To which the answer is, "Yes, that can happen if you change your URLs, but now it's too late to go back..."

Such things make me sad, because a little research before making such a drastic change would have saved a lot of grief -- and in some cases, it might have saved a company from going bankrupt from lack of Web traffic after changing all their URLs and losing all of their PageRank and TrustRank.

It is more than just linked pages that are inter-connected on the Web; The operation of a server can and does have drastic effects on the search engine ranking of pages on a site, and 'everything matters' unless the documentation (server, scripting language, shopping cart, database, forum, blog, search engine, etc.) says it doesn't.

Anyway, we hope you'll stick around, read (a lot), and enjoy WebmasterWorld.

Jim