how necessary is it to use redirect

how quick will my site recover in Google SERPs?

[roelp]

Dear webmasters,

I have been messing around with robots.txt / .htaccess for the last two days. I have been reading myself up on this forum / seomoz / sebastians-pamphlet.

The issue is that I have SEO-ed my site. It's all new with descriptive links (page_topic.html) instead of recordid=1.

Google, of course, still has my old links in their database. I was trying to remove the old links, but it seems Google doesn't accept: recordid=*.

Anyway, i was thinking over the necessity of this approach. Is it really needed for me to remove all the old links and redirect them to the new pages. (Which is difficult because htaccess can't rewrite from recordid=1 to page_topic.html)

Will Google re-index my site and remove all old links with the help of a customized 404-page? And if so, will it be done in a reasonable period of time (a few weeks)?

Or would it be better for me to redirect after which Google will notice the new pages and remove the old ones automatically? I'd like to stay on the good side of Google by not having the same content twice (making a simple version from the old link recordid=1 which will redirect to the new page).

I would be very much interested in your opinions.

Best regards,

Roel

[jdMorgan]

> ...difficult because htaccess can't rewrite from recordid=1 to page_topic.html

You can use mod_rewrite in .htaccess to 301-redirect the old dynamic URLs to your new static ones. This is not strictly necessary, but will speed up the transition, and (usually) carry over some of the old URLs' PageRank to the new ones, preserving some of the the value from your old inbound links, and preventing users' old bookmarks from being broken.

See this previous thread: Changing Dynamic URLs to Static URLs [webmasterworld.com].

Jim

[roelp]

Thanks for your reply Jim.

I didn't know the PageRank could be preserved like that. I'll update my robots.txt tonight.

I have a new question now. There are about 325 links to be redirected (301). Won't it make the robots.txt too large?

[jdMorgan]

Why are you modifying robots.txt? Do not Disallow the old URLs. If you Disallow them, then spiders will never 'see' the redirects.

Jim

[roelp]

Now you've lost me Jim. I thought I had to add the 325 redirects to robots.txt.

I wasn't planning on disallowing search engines to find old links.

I was wondering whether it would work when I add 325 redirects.

Best regards,

Roel

[roelp]

Hi Jim,

I am sorry. I meant to say: i need to add 325 redirects in the .htaccess file.

I gave it a shot, it didn't work. I tried reading up on posts, but that doesn't seem to help either.

If you have the time, could you look at this and give me a hint of what I am doing wrong?

####ERROR_HEADER####
#ErrorDocument 400
#ErrorDocument 401
#ErrorDocument 402
#ErrorDocument 403
ErrorDocument 404 http://www.example.nl/404.html
#ErrorDocument 500
#ErrorDocument 501
#ErrorDocument 502
#ErrorDocument 503
####ERROR_TAILER####

RewriteEngine on
RewriteCond %{HTTP_HOST} ^example\.nl
RewriteRule ^(.*)$ http://www.example.nl/$1 [R=permanent,L]

RewriteRule ^([^/\.]+)-name-of-link.html$ cat.php?catid=$1 [L]
RewriteRule ^([^/\.]+)-name-of-link.html$ folder.php?recordID=$1 [L]

redirect /winkels/folder.php?recordID=1234 http://www.example.nl/store1-name-of-link.html

Best regards,

Roel

[jdMorgan]

1) You must use a local URL-path for all ErrorDocuments. If you use a full URL, then the result will be a 302 redirect, and the server will not send the correct error code to the client. (See ErrorDocument documentation)

2) You are mixing mod_alias Redirect directives with mod_rewrite RewriteRule directives -- Not a good idea, as you cannot control the order of execution.

3) Mod_alias Redirect directives cannot "see" query strings, so they won't work for you in this way.

4) Your RewriteRules are not in the correct order -- Do external redirects first, in order from most-specific to least-specific, then do internal rewrites. Your non-www to www domain redirect should be the last redirect. This avoids multiple 'stacked' redirects for a single client request.

5) If your mod_alias Redirect directives *did* work, then they would have interacted with your RewriteRules, and created an 'infinite loop' or a 'dead-loop'. This is because there is no test to see if the dynamic URL was requested by the client, or if it was the result of your internal rewrite of a static URL.

I'd suggest:

#### Define Custom Error Documents ####
# ErrorDocument 403 [b]/403.html[/b]
ErrorDocument 404 [b]/404.html[/b]
# ErrorDocument 410 [b]/410.html[/b]
#
# Enable Rewrite Engine
RewriteEngine on
#
# Externally redirect only direct client requests for dynamic URLs to static URLs
#
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /winkels/cat\.php\?recordID=1234\ HTTP/
RewriteRule ^winkels/cat.php$ http://www.example.nl/cat1-name-of-[b]cat[/b]-link.html? [R=301,L]
#
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /winkels/folder\.php\?recordID=4567\ HTTP/
RewriteRule ^winkels/folder.php$ http://www.example.nl/store1-name-of-[b]folder[/b]-link.html? [R=301,L]
#
# If non-canonical domain request not already corrected by one of the above rules, redirect it
RewriteCond %{HTTP_HOST} ^example\.nl
RewriteRule (.*) http://www.example.nl/$1 [R=301,L]
#
# Internally rewrite static URLs to appropriate script
RewriteRule ^([^/.]+)-name-of-[b]cat[/b]-link.html$ /winkels/cat.php?catid=$1 [L]
RewriteRule ^([^/.]+)-name-of-[b]folder[/b]-link.html$ /winkels/folder.php?recordID=$1 [L]

Notice that there must be *some* difference in the 'fixed' part of the URL-paths examined by the last two RewriteRules above. Otherwise, only the first of them will ever be applied.

Also, I suggest that you work with this small number of rules until you test them and get them working perfectly. Then add more. :)

Jim

[edited by: jdMorgan at 4:27 pm (utc) on Aug. 7, 2008]

[roelp]

Sorry for getting this post on top again, but I need to give a big thank you to Jim!

Your tips really helped me out and it's all working like a charm.