Closing down a prominent section on my site - how to handle redirects, etc?

[contentmaster]

I have an informational HTML website that also offers a niche service. The domain is about 7 years old and the website has done rather well.

With time, the overall focus of the website has changed and I want to stop / shut down the service which we offered, while continuing with the other major part of the website (information). The service section has about 20-30 webpages.

I want to (ideally) delete all the content and remove those pages since they are no longer relevant to the website. However, I'm worried about how this will impact the overall website since these pages may be listed elsewhere on the web.

1)I am planning to migrate the website to WordPress and only keep the relevant pages on the "new" site. Say I delete all the pages and remove links to those sections from my entire website. How will this impact my search rankings & the website in general?

2) Is there a better way of handling this, something on the lines of redirecting those pages to the home page or a general page that talks about the website's new focus and discontinuation of the service?

3) Has anyone done this before?

Your inputs will be highly appreciated.

[netmeg]

Yes I've done it, but I can't guarantee you your overall rankings won't suffer. I would redirect everyone to the page about shutting down the service and the new direction you're taking, myself. But if you had nice juicy backlinks to those pages you're deleting, you'll be giving those up.

You should be able to look in GWT and see how much traffic (and what kind of ranking) the soon-to-be-removed pages are giving you now, and gauge what effect it may have.

[contentmaster]

Thanks for your reply.

In case, I do delete those sections/pages and redirect them to one page that talks about the site's new direction, will Google rankings settle with time?

Any other ideas on how I can manage this?

[Mentat]

1. Be very clear if you want to redirect or delete.
301 redirect is only for the same content moved to another location!

2. If you delete the pages, be sure to return error 410 (Gone forever), not 404

3. You can also use the Remove URL function in WMT.

[lucy24]

something on the lines of redirecting those pages to the home page

Nooo!

Is this the first time you've shut down a part of the site? If so, the mechanics will be absurdly easy. Probably two lines in htaccess (assuming Apache): one to return a 410 for this directory, another to set up a 410 ErrorDocument. The error document itself can be specific to the newly-shut-down area. That way, the 410 response itself is all you need.

Personal experience suggests that the googlebot stops crawling a lot faster when it meets a 410 instead of a 404. Other search engines ("#2 We Try Harder") don't seem to care.

[contentmaster]

Be very clear if you want to redirect or delete. 301 redirect is only for the same content moved to another location!

I want to completely remove / delete the section and its pages.

If you delete the pages, be sure to return error 410 (Gone forever), not 404.

Currently, I am planning to use the Redirection tool to handle all the redirects after migration to WordPress. Can I use it to return error 410?

You can also use the Remove URL function in WMT.

Since there are more than 30 webpages, should I put in all these URLs through WMT?

[lucy24]

Can I use it to return error 410?

You'd have to ask someone who speaks WordPress. WP is built on mod_rewrite within Apache. The [R] flag is nominally Redirect-- hence the R-- but can actually be used to return absolutely any response in the 3,4,5xx range.

If you're removing an entire directory, you can do it all at once in wmt. There's a popup.

You said at the beginning that you're "planning to migrate" to WP. That implies you're not there yet. No point in moving material that you're about to discontinue.

[Andem]

If the content is still useful, or as you put it, prominent, then why remove it at all? I think you have been reading too many paranoid posts here on WW.

If you do eventually decide to completely remove the content, I would have to echo Mentat's post; 410s are probably the best way. Although if you can't do 410s, then 404s should also serve that purpose. I wouldn't suggest 301'ing them to non-related pages or your homepage.

[contentmaster]

If the content is still useful, or as you put it, prominent, then why remove it at all?

Like I said, we are changing the overall focus of the website and want to shut down the service we were offering. Hence, keeping those pages does not make sense. Since the website will get a fresh breath of life with WP, I want to keep content that is relevant as we go forward.

I wouldn't suggest 301'ing them to non-related pages or your homepage.

Makes sense, thanks.

No point in moving material that you're about to discontinue.

I do not want to move those 20+ pages, rather just delete them and not have any penalty when Google looks for those indexed pages. Setting up 410s should take care of that, right?

[not2easy]

If you have the time, you can noindex the pages in that directory prior to deleting them or serving a 410. Be sure to allow crawling (don't disallow in robots.txt) until Google knows they are not to be indexed or they never quit coming back to look for them. Just a lot of 404s to deal with that can be avoided. The order I would try to work it is:
1. remove all links pointing to that directory
2. noindex
3. Remove Directory (in GWT)
4. return a 410
But it depends on your timetable.

[contentmaster]

Thanks. I'm willing to give it the time it requires as there are a lot of pages and I want to get it done the correct way.

Having said that, I have another site that has been moved to WordPress. There were a few pages that I deleted (didn't include in the new site and redirected those pages to the home page. I realise now that was a mistake. Can you please guide me as to how to correct this mistake?

[JesterMagic]

If I were you I would remove the pages first (and return a 410) and wait a month or so to see how things play out with the rankings. If things remain stable you can then migrate to WordPress. I personally wouldn't do both at the same time since if you have ranking issues you won't know if it is because of the missing pages (and the backlinks those pages had) or because you migrated to WordPress.

[JD_Toims]

Why not?

<div id="overlay" style="position:absolute;z-index:10;top:100px;left:50%;width:400px;margin:0;margin-left:-200px;height:200px;background:rgba(0, 0, 0, 0.7);border:0;border-radius:8px;color:#F5F5F5;"onClick="closeOverlay()">
Sorry, but we've decided to focus on [blah] and unfortunately, that's led us to need to stop [blah-2-ing].

We've left our information about [blah-2] here for you as well as providing you with some links to others who still [blah] below and at the conclusion of our page on [blah-2].

Simply click the X or the word close above to view our information or click one of the links below to visit some of those who currently [blah-2].
</div>
</body>

Something like the above gives people who are looking for [blah-2] the knowledge you now [blah] instead and someday they may be interested in [blah], so why not let them know you do it every chance you get, even if they might not need it today?

[not2easy]

RE:

There were a few pages that I deleted (didn't include in the new site and redirected those pages to the home page. I realise now that was a mistake. Can you please guide me as to how to correct this mistake?

Pages that don't exist should return a 404, redirecting them to an unrelated page like the home page gives you a "Soft 404" - something Google does not like. A 404 is normal, a Soft 404 is not normal. If the pages don't exist any more, let them return a 404 error, no penalty for that.

[contentmaster]

If I were you I would remove the pages first (and return a 410) and wait a month

I can probably try doing that for the website that's yet to be migrated.

If the pages don't exist any more, let them return a 404 error, no penalty for that.

Okay, so now I have removed the redirects to the home page for these 20+ pages which means that they will automatically return a 404 error.

I like the idea of customizing the 404 error page so that it offers some value to the visitors (links to other resources).

Why not?

How and where do I add the code you've mentioned. FYI: I'm using a redirection plugin which shows me a list of 404s but I cannot see any options for customising these pages.

Thanks all for your guidance!

[not2easy]

You're using WP, right? Most themes have a 404.php that can be edited. I would add the alternative information there. There is a section at the WordPress Codex site with information on how to create and implement your own 404 page in the rare case that the Theme you are using does not include one. At the WP support site, look for "Custom 404" and you'll find it.

[JD_Toims]

How and where do I add the code you've mentioned.

You can put it anywhere, but I'd put it right before the </body> tag or somewhere below the footer - You need a bit of javascript to close the div, but it's simple.

I had a few minutes so in thinking about it and testing rather than just coding off the top of my head and giving a "here's a starting point" example, there are some adjustments I made below.

<div id="overlay" style="position:absolute;z-index:10;top:100px;left:50%;width:400px;margin:0;margin-left:-200px;padding:10px;height:210px;background:rgba(0, 0, 0, 0.8);border:0;border-radius:8px;color:#F5F5F5;">
<p style="witdh:40px;float:right;margin-top:-8px;"><a href="javascript:void(closeOverlay());" style="text-decoration:none;font-size:10px;color:#D3CDCD;">Close X</a></p>
<p style="width:350px;margin-top:0;margin-right:10px;float:left;">Sorry, but we've decided to focus on [blah] and unfortunately, that's led us to need to stop [blah-2-ing].</p>

<p style="clear:both;">We've left our information about [blah-2] here for you as well as providing you with some links to others who still [blah] below and at the conclusion of our page on [blah-2].</p>

<p>Simply click the X or the word close above to view our information or click one of the links below to visit some of those who currently [blah-2].
</p>
</div>

<script>
function closeOverlay() {
document.getElementById("overlay").style.display='none';
}
</script>
</body>

Should be close -- Throw it on a test page and have a look at it. There are definitely more "elegant" ways to code it [especially the styles as I'm sure Lucy24 will point out ;) lol and "gracefully degrading/upgrading" for those with/without JavaScript enabled, respectively - but it's a decent "get the job done for most people most [+95%] of the time" example] and the style code/spacing may not be right depending on the font size, standard <p> margins/padding, etc. you use but it should be close enough to only need to adjust a bit -- You can even just throw it on a blank .html page and open it in a browser to see what it does.

[contentmaster]

You're using WP, right? Most themes have a 404.php that can be edited.

Yes, I am. Thanks, it worked out perfectly.

[contentmaster]

You can put it anywhere, but I'd put it right before the </body> tag or somewhere below the footer - You need a bit of javascript to close the div, but it's simple

Tried out the code on a blank.htm page like you suggested and it works well - displays a blank page with a popup box that talks about the upcoming change.

I still have a few doubts.
I do not plan to create these pages in WordPress since the section is going to be discontinued. In that case, on which pages should I be putting the code? - In the HTML pages currently there?

return a 410

Is there a easier way to do this compared with .htaccess changes?

The current .htaccess says -

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress

Thanks

[tangor]

The web changes constantly. Pages come, pages go. The best pages GONE are those actually marked 410.

There is something to be said for keeping everything, even stuff you deprecate. Add a single line in Large Bold to those pages "Maintained for archival purposes. Deprecated." You lose nothing.

[contentmaster]

Add a single line in Large Bold to those pages "Maintained for archival purposes. Deprecated." You lose nothing.

Do you mean delete all the content and add this single line? Can downgrading pages (so to speak) like this cause problems for the others valuable pages and the website, in general?

[not2easy]

Pretty sure the idea is to actually maintain the old content, but to let users know it is maintained for archival purposes and that its content is not current. Not appropriate for everything but helpful if it has old inbound links or contains information that brought traffic.

IF you have replaced the content with a new page or new content, it would be helpful for visitors to have a link to the newer content right there - useful to search engines also.

The line by itself in place of the old content would not benefit any user who came in for the old content.

[tangor]

not2easy clarified my intent. You leave the old content up (it is a landing page with link juice). You tell folks (that new line in Large Bold I suggested) lets them know this is not current for your website. AND, your page still has links to your new site content so the user can find you. Again, nothing lost, one line added, traffic can continue.

IF, however, THE PAGES MUST BE DELETED because they are no longer accurate or have any value whatsoever, then just delete them and return a 410 gone via .htaccess. Best method and does not beg any hits from the search engines.

[Robert Charlton]

"Maintained for archival purposes. Deprecated." You lose nothing.

IMO, there is a potential loss here. If it's a landing page, this approach may risk sending searchers to click their back buttons, to return to Google for the same or similar search. You don't want that to happen.

Your choice very much depends on the nature of the material and how related the service and the information are....
a) would someone looking for the service also be interested in your information, either now or at some later time?
b) would you be doing the searcher a favor sending them to current providers of the same service?

If you do send the searcher to other providers, btw, take pains not to make Google think that these links might be ads or affiliate links of some sort, or that you're funneling or deceptively redirecting traffic or link juice. The kind of wording that JD_Toims suggests would IMO keep you safe, and would accomplish both (a) and (b)...

We've left our information about [blah-2] here for you as well as providing you with some links to others who still [blah] below and at the conclusion of our page on [blah-2].

Definitely check out whether there's enough inbound link juice and traffic to these pages to concern yourself about preserving it.

[contentmaster]

a) would someone looking for the service also be interested in your information, either now or at some later time?
b) would you be doing the searcher a favor sending them to current providers of the same service?

I would prefer to point towards other useful resources on my website rather than pointing them to other websites and then worrying about the quality of content on these 3rd party sites over a period of time.

[contentmaster]

I am ready to move to WP. The pages I'd like to move forward with are ready in a subdomain. The ones I want to say goodbye to are continuing in htm format on the current site. I was wondering if anyone can help me with the .htaccess code for the 410 (GONE FOREVER) pages.

FYI: I have several folders with many webpages for which I want to return the 410 code.

From what I understand, there should be a 410.html page with the redirect content (search box, sitemap). Once that is done, what should I add to the current code so it returns 410 for all the folders & their pages?

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress

[JD_Toims]

Usually the Apache Forum [webmasterworld.com] is a better place to get answers, and they'll help you out there if you decide to fix the garbage mod_rewrite provided by WP, but for the 410's you can simply:

RewriteEngine On
RewriteBase /
RewriteRule ^(?:folder-1|directory-2|some-other-directory)/ - [G]
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]

* Note: My example could be made more efficient in many cases, but not as much more as the WP provided code could be.