Why isn't .htaccess working properly to redirect old to new pages - #2

[KallenWeb]

I thought I'd make a new post, hopefully in the proper format and following all the posting rules properly (hopefully).

This is my current "oldsite.com" .htaccess:

Options -Indexes

RewriteEngine On

# old homepage to new homepage
redirect 301 / https://www.newsite.com/
redirect 301 /index.html https://www.newsite.com/index.html

# old site pages to new site pages with index
redirect 301 /shop-garden-store/index.html https://www.newsite.com/garden-center-kalamazoo/garden-store.html
redirect 301 /fence-and-railing/index.html https://www.newsite.com/fence-gate-railing-services.html
redirect 301 /landscape-arborist-services/index.html https://www.newsite.com/arborist-services-kalamazoo/arborist-services.html
redirect 301 /schedule-free-estimate/index.html https://www.newsite.com/request-a-proposal.html
redirect 301 /arborist-evaluation/index.html https://www.newsite.com/arborist-services-kalamazoo/arborist-services-contact.html
redirect 301 /get-to-the-store/index.html https://www.newsite.com/contact-us.html
redirect 301 /contact-us/index.html https://www.newsite.com/contact-us.html
redirect 301 /my-lawn-planner/index.html https://www.newsite.com/garden-center-kalamazoo/garden-store.html
redirect 301 /fence-and-railing/vinyl-fencing/index.html https://www.newsite.com/residential-fence-kalamazoo/residential-vinyl-fencing.html
redirect 301 /fence-and-railing/aluminum-fencing/index.html https://www.newsite.com/residential-fence-kalamazoo/residential-ornamental-aluminum-fencing.html
redirect 301 /fence-and-railing/western-red-cedar/index.html https://www.newsite.com/residential-fence-kalamazoo/residential-wood-fencing.html
redirect 301 /fence-and-railing/treated-pine/index.html https://www.newsite.com/residential-fence-kalamazoo/residential-wood-fencing.html
redirect 301 /fence-and-railing/chain-link/index.html https://www.newsite.com/residential-fence-kalamazoo/residential-chain-link-fencing.html
redirect 301 /fence-and-railing/vinyl-coated-chain-link/index.html https://www.newsite.com/residential-fence-kalamazoo/residential-chain-link-fencing.html
redirect 301 /fence-and-railing/automated-gate-systems/index.html https://www.newsite.com/residential-fence-kalamazoo/gate-operator-systems.html
redirect 301 /fence-and-railing/vinyl-railing/index.html https://www.newsite.com/residential-fence-kalamazoo/vinyl-railing.html
redirect 301 /fence-and-railing/aluminum-railing/index.html https://www.newsite.com/residential-fence-kalamazoo/aluminum-railing.html
redirect 301 /workspace/pages/nw-map.html https://www.newsite.com/contact-us.html
redirect 301 /workspace/pages/portage-map.html https://www.newsite.com/contact-us.html
redirect 301 /workspace/pages/sw-map.html https://www.newsite.com/contact-us.html
redirect 301 /http://farmngarden.com/workspace/content/What's%20on%20Sale%205%2020%20to%205%2025.pdf https://www.newsite.com/garden-center-kalamazoo/garden-store-weekly-specials.html

# old site pages to new site pages without filename or trailing /
redirect 301 /shop-garden-store https://www.newsite.com/garden-center-kalamazoo/garden-store.html
redirect 301 /fence-and-railing https://www.newsite.com/fence-gate-railing-services.html
redirect 301 /landscape-arborist-services https://www.newsite.com/arborist-services-kalamazoo/arborist-services.html
redirect 301 /schedule-free-estimate https://www.newsite.com/request-a-proposal.html
redirect 301 /arborist-evaluation https://www.newsite.com/arborist-services-kalamazoo/arborist-services-contact.html
redirect 301 /get-to-the-store https://www.newsite.com/contact-us.html
redirect 301 /contact-us https://www.newsite.com/contact-us.html
redirect 301 /my-lawn-planner https://www.newsite.com/garden-center-kalamazoo/garden-store.html
redirect 301 /fence-and-railing/vinyl-fencing https://www.newsite.com/residential-fence-kalamazoo/residential-vinyl-fencing.html
redirect 301 /fence-and-railing/aluminum-fencing https://www.newsite.com/residential-fence-kalamazoo/residential-ornamental-aluminum-fencing.html
redirect 301 /fence-and-railing/western-red-cedar https://www.newsite.com/residential-fence-kalamazoo/residential-wood-fencing.html
redirect 301 /fence-and-railing/treated-pine https://www.newsite.com/residential-fence-kalamazoo/residential-wood-fencing.html
redirect 301 /fence-and-railing/chain-link https://www.newsite.com/residential-fence-kalamazoo/residential-chain-link-fencing.html
redirect 301 /fence-and-railing/vinyl-coated-chain-link https://www.newsite.com/residential-fence-kalamazoo/residential-chain-link-fencing.html
redirect 301 /fence-and-railing/automated-gate-systems https://www.newsite.com/residential-fence-kalamazoo/gate-operator-systems.html
redirect 301 /fence-and-railing/vinyl-railing https://www.newsite.com/residential-fence-kalamazoo/vinyl-railing.html
redirect 301 /fence-and-railing/aluminum-railing https://www.newsite.com/residential-fence-kalamazoo/aluminum-railing.html

# old site pages to new site pages without filename but with trailing /
redirect 301 /shop-garden-store/ https://www.newsite.com/garden-center-kalamazoo/garden-store.html
redirect 301 /fence-and-railing/ https://www.newsite.com/fence-gate-railing-services.html
redirect 301 /landscape-arborist-services/ https://www.newsite.com/arborist-services-kalamazoo/arborist-services.html
redirect 301 /schedule-free-estimate/ https://www.newsite.com/request-a-proposal.html
redirect 301 /arborist-evaluation/ https://www.newsite.com/arborist-services-kalamazoo/arborist-services-contact.html
redirect 301 /get-to-the-store/ https://www.newsite.com/contact-us.html
redirect 301 /contact-us/ https://www.newsite.com/contact-us.html
redirect 301 /my-lawn-planner/ https://www.newsite.com/garden-center-kalamazoo/garden-store.html
redirect 301 /fence-and-railing/vinyl-fencing/ https://www.newsite.com/residential-fence-kalamazoo/residential-vinyl-fencing.html
redirect 301 /fence-and-railing/aluminum-fencing/ https://www.newsite.com/residential-fence-kalamazoo/residential-ornamental-aluminum-fencing.html
redirect 301 /fence-and-railing/western-red-cedar/ https://www.newsite.com/residential-fence-kalamazoo/residential-wood-fencing.html
redirect 301 /fence-and-railing/treated-pine/ https://www.newsite.com/residential-fence-kalamazoo/residential-wood-fencing.html
redirect 301 /fence-and-railing/chain-link/ https://www.newsite.com/residential-fence-kalamazoo/residential-chain-link-fencing.html
redirect 301 /fence-and-railing/vinyl-coated-chain-link/ https://www.newsite.com/residential-fence-kalamazoo/residential-chain-link-fencing.html
redirect 301 /fence-and-railing/automated-gate-systems/ https://www.newsite.com/residential-fence-kalamazoo/gate-operator-systems.html
redirect 301 /fence-and-railing/vinyl-railing/ https://www.newsite.com/residential-fence-kalamazoo/vinyl-railing.html
redirect 301 /fence-and-railing/aluminum-railing/ https://www.newsite.com/residential-fence-kalamazoo/aluminum-railing.html

This is the "newsite.com" .htaccess:

Options -Indexes

RewriteEngine On

#use https
RewriteCond %{HTTPS} off
RewriteCond %{REQUEST_URI} !^/\.well-known/acme-challenge/[0-9a-zA-Z_-]+$
RewriteCond %{REQUEST_URI} !^/\.well-known/cpanel-dcv/[0-9a-zA-Z_-]+$
RewriteCond %{REQUEST_URI} !^/\.well-known/pki-validation/[A-F0-9]{32}\.txt(?:\ Comodo\ DCV)?$
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]

#use www
RewriteCond %{HTTP_HOST} ^newsite.com [NC]
RewriteCond %{REQUEST_URI} !^/\.well-known/acme-challenge/[0-9a-zA-Z_-]+$
RewriteCond %{REQUEST_URI} !^/\.well-known/cpanel-dcv/[0-9a-zA-Z_-]+$
RewriteCond %{REQUEST_URI} !^/\.well-known/pki-validation/[A-F0-9]{32}\.txt(?:\ Comodo\ DCV)?$
RewriteRule ^(.*)$ https://www.newsite.com/$1 [L,R=301,NC]

ErrorDocument 404 https://www.newsite.com/404.shtml

The homepage redirects fine. The individual subpages end up with a 404. I'm still stumped.

You can see try the actual links to see the results in a google search here. I am using a url shortener only because I do not want to skirt any rules and the site url is in the google search url: <snip>

Can anyone tell me what I am doing wrong, probably in the oldsite .htaccess, that is resulting in a newsite 404 instead of going to the correct pages? Thank you!

[penders]

redirect 301 / https://www.newsite.com/

Because your first rule redirects everything - the remaining Redirect directives are effectively bypassed. The directives need to be in order of most specific to least specific.

The mod_alias Redirect directive is prefix-matching. So, the above "catch-all" redirect, matches "/anything" and redirects to "https://www.newsite.com/anything" (everything after the match is copied to the end of the target URL). You should have been able to see this in the redirected URL?

Alternatively, if you only want to redirect requests for the document root, then you need to use a RedirectMatch directive instead. Which matches according to a regex and is not prefix-matching. For example:

RedirectMatch 301 ^/$ https://www.newsite.com/

You will need to clear your browser cache before testing as the erroneous 301 (permanent) redirects are likely to have been cached by the browser. For this reason, it is preferable to first test with 302 (temporary) redirects, that are not cached (by default).

ErrorDocument 404 https://www.newsite.com/404.shtml

Bit of an aside, but by specifying an absolute URL in the ErrorDocument directive, this will result in an external redirect to your error document, which is generally undesirable (apart from serving a 3xx response, you lose details about the request that caused the error). You should use a root relative URL instead. For example:

ErrorDocument 404 /404.shtml

Now, the error document will be served via an internal subrequest.

[lucy24]

redirect 301 / https://www.example.new/
redirect 301 /index.html https://www.example.new/index.html

Say what now? Surely those are the same page and should therefore only be reachable by a single URL. That's assuming you really meant example.new/$ and not, as penders explains above, everything on the old domain. But either way, what's the "index.html" doing there? Use this redirect as the opportunity to get rid of all URLs in "index.html" once and for all.

All patterns involving /index.html should optimally be expressed as
RedirectMatch blahblah(/(index\.html)?)?$ etcetera
(where the target is expressed as blahblah/ with final slash)
because search engines will request all three forms:
/directory/
/directory
/directory/index.html
and you want them all to get nicely redirected in a single step.

[phranque]

This is my current "oldsite.com" .htaccess:

you could condense those three sets of redirects into one if you were using mod_rewrite.
you could replace these three rules:

redirect 301 /shop-garden-store/index.html https://www.newsite.com/garden-center-kalamazoo/garden-store.html
redirect 301 /shop-garden-store https://www.newsite.com/garden-center-kalamazoo/garden-store.html
redirect 301 /shop-garden-store/ https://www.newsite.com/garden-center-kalamazoo/garden-store.html

with this:

RewriteRule ^shop-garden-store(/(index\.html)?)?$ https://www.newsite.com/garden-center-kalamazoo/garden-store.html [R=301,L]

[phranque]

This is the "newsite.com" .htaccess:

you should combine the hostname canonicalization redirect rulesets to avoid chained redirects:

RewriteCond %{HTTPS} off [OR]
RewriteCond %{HTTP_HOST} !^(www\.newsite\.com)?$ [NC]
RewriteCond %{REQUEST_URI} !^/\.well-known/acme-challenge/[0-9a-zA-Z_-]+$
RewriteCond %{REQUEST_URI} !^/\.well-known/cpanel-dcv/[0-9a-zA-Z_-]+$
RewriteCond %{REQUEST_URI} !^/\.well-known/pki-validation/[A-F0-9]{32}\.txt(?:\ Comodo\ DCV)?$
RewriteRule ^(.*)$ https://www.newsite.com/$1 [R=301,L]

[lucy24]

if you were using mod_rewrite.

Or by replacing Redirect with RedirectMatch.

If the entire content of example.old is moving to example.new, it should be possible to do everything in mod_alias (Redirect, or RedirectMatch). But if anything at all is staying behind, there will be a need for mod_rewrite, and then everything should change to RewriteRule.

RewriteRule ^shop-garden-store(/(index\.html)?)?$ https://www.example.new/garden-center-kalamazoo/garden-store.html [R=301,L]

Come to think of it, if there is nothing in the form
/shop-garden-store/more-stuff-here-other-than-index.html
then the pattern (either RedirectMatch or RewriteRule) can simply be

^shop-garden-store

without closing anchor.

[phranque]

But if anything at all is staying behind, there will be a need for mod_rewrite, and then everything should change to RewriteRule.

if nothing is staying behind, then oldsite.com should simply be another virtual host on the newsite.com server.
then you can use a single .htaccess file to handle the legacy redirects and the hostname canonicalization redirect.
(which would require that all redirect directives use mod_rewrite)
thus connecting the dots for why i suggested using RewritRule.

[phranque]

RewriteEngine On
...
redirect 301 / https://www.newsite.com/
...

btw "RewriteEngine On" is a mod_rewrite directive.
if you are only using mod_alias directives, it is irrelevant and unnecessary.