How can I manage/clean 301, 410 redirects? Any help guys?
We have a large Website with around 3lakh pages. As such we have a huge list of redirects that we performed at various points of time.
Now I need to manage/clean those URLs. I don't know how long I can keep a redirect? Or the best way to manage redirects?
Can you help me? Thanks a ton!
tedster
4:56 pm on Jun 18, 2012 (gmt 0)
Welcome to the forums, shaunm.
A 410 means the URL is gone - it's not a redirect. If the URL is truly gone, then you don't need to do anything about it.
If maintaining a lot of 301 redirects is a technical burden, you could always drop those that are not showing up in your logs anymore. If no one is using them, then they don't have much use. However, I would also check backlinks for those redirected URLs before dropping tbem - just to make sure that there is no important link juice being captured by the redirect.
If there is no technical reason for dropping the redirects, there's no reason not to keep them in place indefinitely that I can see.
goodroi
5:15 pm on Jun 18, 2012 (gmt 0)
As Ted says, look at your log files and kill redirects that haven't been used in the last 3 months. You don't even need to check backlinks since Googlebot traffic should show up in your log file. If Googlebot hasnt hit that redirect in 3 months, it is unlikely to be passing significant link juice.
shaunm
7:13 am on Jun 20, 2012 (gmt 0)
@Tedster, Goodroi
Thanks a lot guys!
Can you please tell me how/where do I find this so called log files? We are using IIS server..is that found in the server?
Best,
tedster
6:44 pm on Jun 20, 2012 (gmt 0)
On an IIS server, the log files need to be configured in order to do their job properly and there are a good number of options to understand. Using your log files well is a real plus for SEO, so it's good that you are looking into this.
For more help, you can ask more technical questions in our Windows IIS [webmasterworld.com] forum.
lucy24
8:32 pm on Jun 20, 2012 (gmt 0)
We have a large Website with around 3lakh pages. As such we have a huge list of redirects that we performed at various points of time.
One thing you can do is look closely at the wording of the existing redirects. I don't speak IIS so I won't offer details. But you can do a lot with a well-designed Regular Expression. So instead of 3,00,000* individual Redirect lines-- which would slow any server to a crawl-- you do it with a few careful captures.
* Yes, I realize that's your site size, not your number of redirects. But the numbers tend to be proportional, unless you've got a brand-new site where nothing has had time to change.
shaunm
7:35 am on Jun 21, 2012 (gmt 0)
@Tedster
Many thanks for all your help :)
shaunm
7:40 am on Jun 21, 2012 (gmt 0)
@lucy24
I would definitely look into that, thanks )))
phranque
9:09 am on Jun 21, 2012 (gmt 0)
welcome to WebmasterWorld, shaunm!
what you need is called ISAPI_Rewrite which will give you capabilities similar to the apache mod_rewrite module.
tedster
1:02 pm on Jun 21, 2012 (gmt 0)
Well, it depends on what the site's needs actually are.
ISAPI_Rewrite is a third party (paid) module that gives IIS servers something very much like Apache's .htaccess functionality. But in this case, it sounds like the redirects are already in place without any third party module. Recent versions of the IIS server are natively quite improved in the area of redirects, so it's certainly possible.
Redirects may also be scripted for a particular URL with Visual Basic, for example. So for these reasons and more, a more technical discussion in our IIS Forum may be the best idea.
shaunm
1:07 pm on Jun 21, 2012 (gmt 0)
@ Tedster
Yes! We are using a .config file for the redirects through the "regular expressions" I don't think we will need any 3rd party tool.
@phranque Anyway thanks for that buddy!
Jez123
12:36 pm on Jun 22, 2012 (gmt 0)
@ Tedster.
I have a question on 301 redirects. Do I need to keep the actual files (the html pages in my case) on the server once they are redirected or can they be deleted? I have always been led to believe that the actual files would need to be present but I guess that wouldn't be possible if content were database generated would it?
tedster
12:51 pm on Jun 22, 2012 (gmt 0)
When a redirect kicks in at the server level the minute a URL request is received, then no user-agent ever sees the content that used to be served from that URL. So no, you don't need to keep that content hanging around.
On the other hand, sometimes people generate a redirect from a script that is served from the URL itself. I've seen this in PHP on a LAMP stack as well as with various scripting languages on Windows servers. In that case, clearly you still need the script to run from the URL itself.
Sometimes in this scenario I still see the old content left in place in the source code after the script appears. I assume that crawlers can retrieve the old content in that situation. I don't think doing that makes sense either, but some developers still choose to do it.
