Separate DNS Hosting

We need redundancy

[wingslevel]

I host my own sites on several boxes co-located at host #1. It has been a rough summer - between after effects of the blackout, router problems at my host and his provider, and problems with my own coldfusion server, we have been down far too much (including now).

Here is my plan - Outsource my dns function (anybody heard of ultradns?) so that when my site is not performing I can either manually or automatically repoint it to a set of backup servers at host #2. Right now my dns server is at location #1, so if it is down I am obviously out of luck.

The only weakness of all of this is that if the dns host is down - this host has got to be close to bulletproof.

Any thoughts?

[claus]

I use one myself. My DNS provider has five nameservers for me, not the minimum two that hosting companies usually have.

As these firms do make a living from this, they're usually quite stable. I don't know UltraDNS but i just read their PDF: "9 multi-continent locations" (which is 7 in US, 1 in London and 1 in Japan). I don't know if you'll have these six locations for your particular nameservers just because the company has these locations, but i think that's what they would like you to think at least.

Don't know about you, but the Oracle, MSN Hotmail, and Forbes namesdropping sounds a bit too expensive to me and there are tons of them out there to choose from, even free ones. Anyway, if you can afford them i don't see anything that should indicate that you should not choose them, they do provide service and all, and that's valuable as well.

Added: If close to bulletproof is your most serious concern, the geographical spread of this company's servers is a good sign.

/claus

[Stretch]

I use zoneedit.com. I've used them for years and never once had a problem. They're free for 5 domains - 2 name servers each or you can purchase more name servers at minimal cost. Zone records are updated every 5 minutes too which is very handy.

Stretch

[wingslevel]

So how do you switch your traffic from host#1 to host#2? Do you just log onto your dns provider and make the switch?

[Stretch]

Yes. They have a web interface that allows me to update records whenever I want.

However, I've been considering my options for redundancy for quite a while and still don't have a good solution. The problem is that (if my understanding is correct) DNS is cached elsewhere, so even if I change my records in the authoritative name server, the cached record is still being used around the net for XX hours (maybe even days). So redundancy through DNS doesn't seem like a perfect 'bulletproof' option.

Zoneedit also offer a dns failover service which, assuming you can keep your servers in sync, might be a solution - but not one I've been able to explore.

Stretch

[claus]

When you are about to make the switch, the last thing you do before it is to set the TTL on your DNS records down to zero (perhaps in a few steps over some days, dependent on the default TTL on your system). That way the name servers of this world knows that a change is about to come and they should watch out for changes on your domain(s). TTL is the Time To Live setting and it influences the caches.

/claus

[Stretch]

Claus, how does that work exactly? Does the TTL instruct other DNS's when to check the authority NS records or does it just update the records on the authority NS?

I've considered having NS1 on webserver1 and NS2 on webserver2 but with the records for each domain pointing to the webserver that the NS resides on. In this way, I figure that any requests for a site on one of my servers would be routed to webserver1 assuming NS1 responded. If NS1 didn't respond then NS2 would pick up the request and point it to webserver2 therefore providing redundancy. Of course, this would only work if caching was not an issue - hence my question.

Actually, I guess my question is twofold:

1) Assuming NS1 responds is it always authoritative or do some requests get sent to NS2/3/etc regardless?

2) With a TTL of (say) 300 seconds would record changes be reflected in that time with other DNS's or is it their TTL that determines this?

I've been pondering this for ages. Now my curiosity is piqued so any advice would be most appreciated.

Stretch

[claus]

I'm not an expert, as the DNS is complicated, but i'll answer as good as i can.

>> I've considered having NS1 on webserver1 and NS2 on webserver2

You have to do this, it's how it works. Or at least it's recommended - even the separate locations are recommended. If NS1 does not respond, NS2 will get the query, then NS3, NS4 and so on.

>> but with the records for each domain pointing to the webserver that the NS resides on

It's generally not a good idea to have your NS and your content (webserver) on the same machine. If one goes down, so goes the other. DNS must be redundant, that's why at least two NS on different machines is recommended. And not on the same machine as the webserver, preferably.

>> NS1 -> WS1 && NS2 -> WS2

I really don't think you can do that in the way you specify. NS1 and NS2 both point your domain name to the same IP address. One is backup for the other (and NS3 is backup for NS2 and so on).

When the request hits your IP it will meet a web server. The DNS does not control if tis webserver is running or not, it just identifies the right webserver to send the request to. To do what you want you need redundant machines (servers, load balancers, routers, whatever) set up at your IP which will direct the incoming traffic to a machine that's on.

That's two levels of redundancy (1) at the DNS level with nameservers, and (2) at the IP level with webservers.

>> Assuming NS1 responds is it always authoritative or do some requests get sent to NS2/3/etc regardless?

All your NS's are authoritative. If NS1 fails, the request goes to 2,3, etc. That's why they have identical setup.

>> With a TTL of (say) 300 seconds would record changes be reflected in that time with other DNS's or is it their TTL that determines this?

There is only one DNS, it's the term for the whole system of nameservers and caches. Your TTL is specified on your nameservers and it tells the other nameservers for how long they shaould cache the records of your domain. Only your NS can specify the TTL for your domain, but of course this can be overruled by cache settings other places. There's just no point in doing this, as these other places would then get invalid records.

>> Does the TTL instruct other DNS's when to check the authority NS records or does it just update the records on the authority NS

The TTL instruct other nameservers on the proper time for caching your records. It instructs other Nameservers on when to check your authoritative NS's. You update the TTL on your Nameserver, nobody else can do this, and your update should be reflected instantly on your own NS - or in no longer than some one digit amount of minutes.

Say that you have a long TTL, lets say a week. Then the other nameservers will cache your records for a week before they come to your NS to look for changes again. If you then change it to a day, then it will not be reflected the next day all over the world, because som nameservers will have checked just befor you made the change, and these will wait a week before they come back.

So a change from one week to one day will take 7 days to propagate (the term for changes to be reflected in the whole system). When you've got it at one day, then the other NS's will check back on a daily basis, and then you can set it to zero the day before you switch, then you will have the NS's of this world buzzing around all the time trying to update, and when you then make the switch it will be reflected instantly (or rather, as fast as it gets).

After the switch, be sure to put it back up on a high level instantly, (the level before the step-down-TTL-process), otherwise you will have a permanent heavy load on your NS from the whole world.

/claus

[Stretch]

Many, many thanks for the very informative reply claus. It's helped me get my head around how to use TTL to my advantage.

Agreed that load balancing etc would be the best solution but sadly my setup is dictated by budgets which don't extend that far.

I'll stick with my current setup for the time being but will continue to try and devise a solution that will automate redundancy (but on a budget).

Thanks again - very helpful.

Stretch