Forum Moderators: phranque
[dnsreport.com...]
Apparently something that has been going on for years has now come to a head and it has to do with DNS Recursion. I'm not a DNS specialist so please bear with my terminology and correct me if I'm wrong, that's the only way I'm going to learn! ;)
The New Error Most Will See
ERROR: One or more of your name servers reports that it is an open DNS server. This usually means that anyone in the world can query it (it is possible that the DNS server advertises that it does recursive lookups when it does not, but that shouldn't happen). This can cause an excessive load on your DNS server. Also, it is strongly discouraged to have a DNS server be both authoritative for your domain and be recursive (even if it is not open), due to the potential for cache poisoning (with no recursion, there is no cache, and it is impossible to poison it). Also, the bad guys could use your DNS server as part of an attack, by forging their IP address. Problem record(s) are:
Server xx.xx.xx.xx reports that it will do recursive lookups.
It appears the change in the DNS Tool report has caused some heated discussion from a few who use the tool regularly. Many do not like to see that FAIL message when they've had an Open DNS Server for as long as they can remember.
To make a long story short, having a DNS server that allows recursion for the Internet is like running an open SMTP relay.
The Good Guys have found over half a million. The Bad Guys will find yours. Bad guys will use spoofed UDP packets to fire-and-forge large DNS requests, and the recursing server will send the fragmented replies to the victim (the forged source of the UDP). There are rumblings by very smart people to "do something about this".
A few days ago, it was pointed out that there was a new attack using open DNS servers as part of a DDoS, using amplification (where sending the open DNS servers would send packets 50 times as large as the ones that were originally sent), making this a very serious issue.
Are you aware of and/or are you doing anything about this? I'm in the process now of discussing all of this with my server administrators and want to make the changes to eliminate that failure on the DNS Report. Anytime I see red on that report, the hair on my neck rises. Many of the issues we see here at WebmasterWorld can be traced back to DNS Issues so it is important that you keep a regular eye on what your DNS is up to! ;)
Brett, congratulations, you passed for this particular test. Over 75% of the sites out there do not, including my own which I am in the process of correcting.
This does not constitute a problem. And this is not something we will be able to change in any case. You can, however, set up your own
nameserver if this still concerns you.
Unfortunately it appears that this type of response is being received by many. These are the hosts that you need to move away from now. And, the bigger they are, the harder they are going to fall.
Another thing, if you get any flack from your host about making this important change, I'd have to start questioning why. In some instances, it is possible that the host is well aware of what is going on. If the network you are hosting on also hosts websites that would attract this type of technical foul play, well, I'll leave the rest to you.
Bottom line. If the host does not correct the issue immediately, they are then part of the problem. You'll need to find a solution before the problem rears it's ugly head. If it hasn't already. :(
In early February 2006, name servers hosting Top Level Domain zones were the repeated recipients of extraordinary heavy traffic loads. Analysis of traffic by TLD name server operators and security experts at large confirmed that DNS packets comprising the attack traffic exhibited characteristics associated with previously attempted DDoS attacks collectively known as amplification attacks.
---
There is no issue as nobody uses our DNS servers to perform outbound lookups. Essentially, if a user is using our DNS servers to query for something they do not have, our servers will continue to query the other servers trying to determine the answer for the querying user instead of redirecting to the other DNS server who has authority over the domain being resolved. Thus, if an attacker can somehow get your server to query their server for a domain they do not control (this is a theoretical security issue mind you :) ), then they can provide a false answer, and your server will CACHE the answer for the query. Thus, anybody who queries your server for that domain, will be given the bad answer because DNS caches the query for performance issues. so, if they could in theory get your server to be the authority for a banking site, then they could redirect all the queries to their own version and steal credit cards. But, since NOBODY should be querying our servers for domains which we do not host, and the only queries that we should be receiving are other servers performing recursive lookups or clients wishing to access the domain hosted on our network, this feature really does not negatively effect our security.
---
There is no issue as nobody uses our DNS servers to perform outbound lookups.
It sounds like they aren't talking about an open DNS server, but instead are talking about cache poisoning.
Their answer might be a fairly reasonable for an argument about cache poisoning (although it doesn't make much sense, as they say "nobody uses our DNS servers to perform outbound lookups" and "NOBODY should be querying our servers for domains which we do not host", which contradicts "if a user is using our DNS servers to query for something they do not have" and "the only queries that we should be receiving are other servers performing recursive lookups").
But it does not address why they have open DNS servers. It sounds like they let their customers use the DNS servers recursively (which isn't best practice, since they also are authoritative DNS servers, but it should work fine if there are no security problems), but still need to set the DNS server to only respond to recursive queries from their own network.
-Scott
It sounds like if I disable recursion I would break my mail servers since Windows DNS doesn’t allow you to have exceptions. So would it best practice to turn off recursion on my 2 authoritative DNS servers and put up a new Windows 2003 DNS server as a caching server for my mail to do lookups against but not allow any inbound (port 53) traffic to it?
Thanks in advance!
It sounds like if I disable recursion I would break my mail servers since Windows DNS doesn’t allow you to have exceptions.
Correct.
So would it best practice to turn off recursion on my 2 authoritative DNS servers and put up a new Windows 2003 DNS server as a caching server for my mail to do lookups against but not allow any inbound (port 53) traffic to it?
Correct. :)
Another option that is simpler in some ways yet more complex in other ways would be to set up a caching DNS server on the mailserver itself. I've done that in the past, and it works quite well. Either setting it up not to accept queries from outside the server or settings up a firewall in front of it to block incoming packets to port 53 will do the trick.
i have posted on the dnsstuff forum today noting that a blanket approach to this issue (by clicking the "disable recursion" check box) does indeed have an effect on SMTP services - my example is failing form-to-mail components ...
i thought it was the "forwarding" element that was being turned off and thereby causing the mail to fail but after reading this, it sounds more like the recursive lookups feature that is required. anyone know of a solution?
sounds like a good idea dnsstuff but having limited knowledge whilst tackling my first dedicated W2003 server, and i'm sure i'm not alone, we could all do with a simple walkthrough of that solution!?
The simplest way would likely be using SimpleDNS Plus (it's what we use for recursive DNS on www.DNSstuff.com, actually).
If you are looking for a free option, you could use Microsoft DNS -- the setup isn't too important just so long as you make sure that the firewall won't allow incoming packets to port 53. To be extra safe, I would recommend not using forwarders -- there is rarely ever a need for them, and there may be problems with using them.
i have posted on the dnsstuff forum today noting that a blanket approach to this issue (by clicking the "disable recursion" check box) does indeed have an effect on SMTP services - my example is failing form-to-mail components ...
Correct. That's because you are using recursion, and that's a big problem with Microsoft DNS -- it doesn't let you enable recursion for just your local network. It's kind of like a mailserver that is always an open relay.
If you need recursion, your main options include [1] running a different type of DNS server (such as SimpleDNS Plus), or [2] having a separate server for recursion.
-Scott
I am presuming, of course, that it is perfectly fine to run 2 DNS servers on the same physical server? can't see why it should conflict ...
That is fine, just so long as they listen on different IPs.
-Scott
"Perils of Transitive Trust in the Domain Name System" by Venugopalan Ramasubramanian and Emin Gün Sirer [usenix.org] from Cornell University (2005-08-15)
The paper provides a fascinating look into DNS trust relationships.
"To all:
Beginning at approximately noon Wednesday May 3rd the Tucows network has been under a severe DDOS (Distributed Denial Of Service) attack whose impact has been amplified by the attack's use of recursive name servers.
The extent of the DDOS attack was enough to knock out two of the three upstream providers to our colocation facility. Because of this, for the first four and a half hours of the attack, it was assumed by all involved that this was a network outage. It was only when the upstream providers were able to recover from the initial blow that we were able to determine that it was in fact an DDOS attack.
The attack, while apparently directed at a single website, had an impact beyond its target making large portions of our network inaccessible for periods of time throughout the day. While the site under attack used our Managed DNS Service, Tucows is not the domain's registrar and as such our options for resolving without impact have been limited. Our operations staff, along with those of both our colocation provider and their upstream providers have been working diligently to return service to normal.
Our operations staff will be working through the night to make this situation as painless as possible. I can only tell you all that I am sorry and we will continue to do everything in our power to make this better."
Just in case anyone thinks this recursive dns issue isn't a real problem, yesterday opensrs (tucows) suffered a massive denial of service. Here are some details as put out by their reseller update.
I had to back away from this particular topic because some of my peers think I'm crazy. Most think that this isn't a problem that affects them and they feel it is such a small issue that it is not worth discussing.
I still hold to all of my original statements and I personally think the storm is brewing (and so do many security experts). But, DG has clearly indicated that the "sky is not falling" so I guess it's nothing to worry about. :(
Most think that this isn't a problem that affects them and they feel it is such a small issue that it is not worth discussing.
Part of the issue is people making their operation work versus making it work according to the proper standards. For example we see a lot the same issues when it comes to mail handling. Service provider buys a mail server program or downloads a free one, gets it installed sends a few test messages, voila it is working, get some clients, starts taking in some revenue. But is that mail server correctly setup?
Is it an open relay?
Does it have proper reverse dns setup?
Do the domains have spf records?
etc etc.
These kinds of issues consume an inordinate amount of time. We get queries from our customers as to why something originating elsewhere isn't working and then we have to track it down and explain to the customer that the other network has this or that flaw. Then if the other network doesn't want to fix it, we are left in a catch 22 situation of do we lower our standards to allow it, or do we risk losing a customer.
Lately, the better course is to adhere to the proper standards.
Just in case people are not aware of it there are RFC's for all of this stuff. RFC stands for Request for Comments and originate from IEEE.
originate from IEEE
or from the IETF, or any number of other standards agencies. ;)
P1, this is exasperating because even if my DNS machines are in tip-top shape and properly configured, there is literally nothing I could do to prevent the sort of event that happened to Tucows from happening to me.
It didn't matter at all that their servers were cool ... it was a problem with intermediary servers over which they had no control, and possibly whose existence they had no knowledge of.
Did you read the white paper I referenced, above? It makes painfully clear the scope of this problem.
Do you think is would be appropriate to pursue some type of global legislation to ram a "cure" down every DNS administrator's throat?
Do you think is would be appropriate to pursue some type of global legislation to ram a "cure" down every DNS administrator's throat?
The legislation will come in the form of a DDoS Attack at which time they will rethink their position. :(
There are a few organizations that are now making reports publicly available on those servers that are subject to these exploits. Once that information gets into the mainstream, I'm sure many will start to take this seriously.
Also, there are a bunch of servers out there in other countries that play a part in all of this. My understanding is that few really care and/or fully understand the magnatude of this.
My understanding is that combining an Authoritative & Recursive DNS can lead to "poisoning the cache" and that is "bad". Creating separate DNS for my hosted sites and the "outside" is a bit beyond me at this time, although I'm trying to get there.
My question is "What effect, if any, is flushing the cache on a daly or hourly basis have?"
Will that delay, deny or prevent the "bad guys" from using my site?
Thanks in advance for any help/response.
Larry
My understanding is that combining an Authoritative & Recursive DNS can lead to "poisoning the cache" and that is "bad". Creating separate DNS for my hosted sites and the "outside" is a bit beyond me at this time, although I'm trying to get there.
Actually, cache poisoning will occur regardless of whether or not the server is authoritative. That is, if your recursive DNS server is vulnerable, the cache can be poisoned whether or not is is also an authoritative DNS server. If it is not vulnerable, then the cache cannot be poisoned whether or not the server is also an authoritative DNS server. So the key is to make sure that you are running a recent version of your DNS software.
One of the reasons why it is best practice to run separate authoritative and recursive DNS servers is that if cache poisoning is possible, you want to be 100% sure that it doesn't affect domains that you are authoritative for.
The issue with open DNS servers can be solved by making sure that your recursive DNS server only is recursive for your local network (which can't be done with Microsoft DNS, though). If your DNS server is not authoritative, the bad guys are less likely to find it (but may!).
In your case, the best option may be to use your Internet provider's recursive DNS servers. If that isn't possible, then you can have the DNS servers be both authoritative and recursive, if you make sure that they are recursive only for your local network.
My question is "What effect, if any, is flushing the cache on a daly or hourly basis have?"
That would minimize the impact of cache poisoning, if your DNS server is vulnerable (but would destroy some to much of the benefit of caching, depending on the frequency of flushing). It would not have any effect on being an open DNS server, though.
-Scott
