Why isn't .htaccess working properly to redirect old to new pages?

[KallenWeb]

We moved a site from example.com to example.new. The pages also changed. We used a simple .htaccess file on example.com to redirect to example.new: (scroll to lower half of this post to view the full URLs).

Options -Indexes
RewriteEngine on
RewriteCond %{HTTP_HOST} ^example.com [NC,OR]
RewriteCond %{HTTP_HOST} ^www.example.com [NC]
RewriteRule ^(.*)$ https://www.example.newer/$1 [L,R=301,NC]

For subpage changes, we have used an .htaccess file on example.newer that redirects to individual new pages.

For example:

redirect 301 /widget-and-handle/index.html https://www.example.newer/blue-widget-handle-services.html
redirect 301 /widget-and-handle https://www.example.newer/blue-widget-handle-services.html
redirect 301 /widget-and-handle/ https://www.example.newer/blue-widget-handle-services.html

This one does not work:
http://www.example.com/widget-and-handle/index.html

Result: https://www.example.newerwidget-and-handle/index.html

This one does work:
http://www.example.com/widget-and-handle

Result: https://www.example.newer/blue-widget-handle-services.html

This one does not work:
http://www.example.com/widget-and-handle/

Result: https://www.example.newer/404.shtml

Even using the redirect checker [redirect-checker.org...] I'm having a very hard time seeing what I could be doing wrong here. Any assistance will be appreciated!

Full url's formatted below to make it easier to read:

We moved a site from example.com to example.new. The pages also changed. We used a simple .htaccess file on example.com to redirect to example.new:

Options -Indexes
RewriteEngine on
RewriteCond %{HTTP_HOST} ^example.com [NC,OR]
RewriteCond %{HTTP_HOST} ^www.example.com [NC]
RewriteRule ^(.*)$ https://www.example.newer/$1 [L,R=301,NC]

For subpage changes, we have used an .htaccess file on example.newer that redirects to individual new pages.

For example:

redirect 301 /widget-and-handle/index.html https://www.example.newer/blue-widget-handle-services.html
redirect 301 /widget-and-handle https://www.example.newer/blue-widget-handle-services.html
redirect 301 /widget-and-handle/ https://www.example.newer/blue-widget-handle-services.html

This one does not work:
http://www.example.com/widget-and-handle/index.html

Result: https://www.example.newerwidget-and-handle/index.html

This one does work:
http://www.example.com/widget-and-handle

Result: https://www.example.newer/blue-widget-handle-services.html

This one does not work:
http://www.example.com/widget-and-handle/

Result: https://www.example.newer/404.shtml

Even using the redirect checker http://www.redirect-checker.org/index.php I'm having a very hard time seeing what I could be doing wrong here. Any assistance will be appreciated! 

[edited by: KallenWeb at 7:25 pm (utc) on Jul 19, 2019]

[edited by: not2easy at 7:52 pm (utc) on Jul 19, 2019]
[edit reason] Exemplified Domains per Charter/ToS [/edit]

[KallenWeb]

This was the code view but I edited my original post instead. Thanks.

[penders]

Have you cleared your browser cache?

Do you have any other directives in the .htaccess file at "www.example.newer"?

[lucy24]

Some observations:

For subpage changes, we have used an .htaccess file on example.newer that redirects to individual new pages.

Why? All redirects should happen in a single step, at the point of the original request.

wrong:
example.com/old-url >> example.new/old-url
example.new/old-url >> example.new/new-url
right:
example.com/old-url >> example.new/new-url

RewriteCond %{HTTP_HOST} ^example.com [NC,OR]
RewriteCond %{HTTP_HOST} ^www.example.com [NC]

Again: why? Do you have other sites sharing an htaccess file with example.old? If not, leave out the RewriteCond and proceed directly to the rule. If yes, it is very unlikely you'll need anything more than the single line

RewriteCond %{HTTP_HOST} example\.com [NC]

without anchors. (Failure to escape the . is a non-lethal error.)

This one does work:
http://www.example.com/widget-and-handle

Is there a CMS involved, OR is that a real physical directory, OR are you doing your own rewriting in addition to the external redirects?

Result: https://www.example.newer/404.shtml

This is probably an entirely separate issue, in which an internal request for your 404 page eventually arrives at the canonicalization redirect, and gets treated just like any other request. (Got a vague feeling I once experimented and found that the [NS] flag does not help.) You need a preliminary rule that says something like

RewriteRule ^40[34]\.shtml - [L]

to keep your ErrorDocuments invisible. If an incorrect or blocked request comes in with the wrong protocol or hostname, leave it that way.

Mixing mod_alias (Redirect by that name) and mod_rewrite (RewriteRule) in the same htaccess is just asking for trouble. You have no control over the order the mods execute--normally it's mod_rewrite before mod_alias--so things may not happen in the order you intended, or things may take two or more steps.

[KallenWeb]

This makes sense. For my next attempt, I am putting all the redirects on the old site hosting .htaccess, and only force www and https (and the 404) on the new site hosting .htaccess.

[KallenWeb]

For future readers:

It ended up being very complicated but in the end I was not able to use

redirect 301 /oldpage/index.html https://www.newsite.com/newpage.html 

AND

redirect 301 /oldpage https://www.newsite.com/newpage.html 

AND

redirect 301 /oldpage/ https://www.newsite.com/newpage.html 

So, since google only had indexed the pages with the trailing "/", I eliminated all the others.

And, then I had to put them long string to short string, so that

redirect 301 /oldpage/innerpage/ https://www.newsite.com/newpage.html 

came before

redirect 301 /oldpage/ https://www.newsite.com/newpage.html 

in the .htaccess file.

Lots more info in this thread if anyone else is struggling with this same issue:

[edited by: phranque at 8:17 pm (utc) on Jul 23, 2019]
[edit reason] snipped links to Google webmaster support threads containing specifics [/edit]

[lucy24]

And, then I had to put them long string to short string

You could also do this with RedirectMatch, allowing you to put $ anchors at the end of the part-to-match.