"DNS propagation" for new domain

Why does it take so long to resolve a new domain?

[iomfan]

This happens all the time:

- I register a new domain with one of my regular registrars who publishes domain data very quickly
- TTL is set to 1800 for the A records (300 for the "mail" subdomain)
- I use several worldwide DNS propagation check sites and a few traceroute servers (in several countries) to confirm that DNS recognises the new domain
- I can see the website I uploaded and successfully send and receive messages via the new domain

Then:
- A DAY LATER a client (using a different provider than myself) tells me that their browser cannot find the new domain

I can understand how, after a domain has been moved to a different server, thanks to domain name information caching on part of PCs and on part of providers, a connection request from a browser may for a while be directed to an old IP address, but I don't understand why a provider cannot find a new domain.

Here is an explanation - copied from another thread - of how the DNS works:

If the information for example.com is not cached locally, the browser sends a request to their DNS server asking for the IP address for example.com.

If that DNS server does not have that information, it forwards the request to a higher tier DNS server. (This may continue for a few more tiers depending on how the respective DNS servers are setup.)

If none of the DNS servers have the information, the request finally gets to a top level DNS server that sends the request to your nameserver.

Your nameserver replies with [the IP address] and the information gets sent back down the chain until it eventually gets back to your browser.

After the initial request, the information is stored at each DNS server for a certain period of time.

Why does this not work in some cases?

[lammert]

Some ISPs use heavy caching of DNS records without looking at the TTL values of the records. Also in some server configurations when a request to an upstream DNS server returns errors (if your client tested the domain name before it went life), further upstream requests for that same query are temporary on hold to prevent excessive traffic.

The longest I have seen personally was a one week delay when switching a website to another server while my TTL records had a value of 3600 seconds. It took one week before the last requests to the old server vanished and all traffic was redirected to the new server.

[phranque]

TTL is set to 1800 for the A records

what was the TTL before being set to 1800 and how long after changing the DNS config did your client see the old IP?

[iomfan]

lammert and phranque, thanks for the comments.

To avoid a misunderstanding: this is about NEW (previously never used) domain names that a given provider's name servers would not know at all when queried for the first time. In that case they are supposed to ask a "higher authority", but that process sometimes takes days - why?

Are some name servers made lazy by having been set to default to "don't know" with a TTL of 86400 or more? Sort of like: "if the client asks often enough the information they are looking for is perhaps important, but if they only ask once we need not bother to serve them?" ;-)

(I understand the "lazy" approach to caching - even ignoring TTL data - and the resultant possible problems when CHANGING servers - if possible I leave the old server configuration online for a week or more after a move. And about your question: my TTL is always 1800, except for the MX where I use 300.)

[phranque]

that sounds like a problem with the client's DNS server, whether it's some internal network appliance or their ISP's.
were all cases the same client/ISP or unrelated?