While pulling HTTP apart, I ran into a step that I had been treating almost as punctuation:
example.com
│
│ DNS
▼
IP address
Browser needs a server.
DNS turns the name into an address.
Onward.
Except that skips over a rather interesting question:
How does my computer actually find out what
example.commeans?
There cannot realistically be one enormous file somewhere containing every name on the Internet.
And even if there were, keeping copies of it current on every machine would be fairly heroic.
So DNS must be doing something more interesting.
The first question is surprisingly simple
At its heart, DNS lets a client ask questions about a domain name.
Something conceptually like:
What is the IPv4 address for example.com?
Or:
Which name servers are responsible for example.com?
Or:
Which mail servers accept mail for example.com?
That immediately improves my mental model.
DNS is not merely:
name → IP address
It is closer to:
name + question
│
▼
DNS
│
▼
answer
The type of question matters.
An IP address is only one kind of information DNS can store.
Names are easier for humans; addresses are useful to networks
When I enter:
https://example.com/
the hostname:
example.com
is useful to me.
The network stack ultimately needs addressing information it can use to communicate with the destination.
DNS sits between those worlds.
Very roughly:
human/application
│
│ example.com
▼
DNS resolution
│
│ address information
▼
network communication
That is already more useful than calling DNS:
the Internet’s phone book.
The phone-book analogy captures the idea of looking something up, but it makes DNS sound like one giant directory.
It isn’t.
DNS is distributed
This seems to be the important architectural idea.
The Domain Name System was designed as a distributed database.
Responsibility for the namespace can be divided between different organisations and servers rather than one machine needing authoritative knowledge of every name.
The namespace itself has a hierarchy.
For:
www.example.com
I can read the labels from right to left:
.
│
com
│
example
│
www
The dot at the top represents the DNS root.
So the fully qualified name can be thought of as:
www.example.com.
with that final dot making the root explicit.
Usually humans leave it off.
DNS doesn’t lose sleep over our typographical laziness.
The hierarchy also allows responsibility to be distributed.
Conceptually:
root
│
▼
.com servers
│
▼
example.com servers
│
▼
information about
www.example.com
The root does not need to know the IP address of every host beneath .com.
It needs enough information to point the lookup towards the servers responsible for the next part of the namespace.
Likewise, the .com infrastructure does not need every record belonging to example.com.
It can identify which name servers are responsible for that domain.
The search can become progressively more specific.
One subtlety: the hierarchy of names does not mean every label is served by a different set of name servers.
Delegation happens at particular points in the namespace. www.example.com, for example, may simply be data inside the example.com zone rather than another independently delegated zone.
That is a cleverer design than:
one DNS server
│
└── knows the entire Internet
which would be an impressively large basket containing an alarming number of eggs.
My application normally doesn’t do all of this itself
Suppose my program wants to connect to:
example.com
It usually does not start manually interrogating root servers.
There are resolver components in between.
At the client end there is commonly a stub resolver, usually exposed to applications through operating-system or library name-resolution functions.
That resolver can send a DNS query to a configured DNS server which performs the heavier resolution work on the client’s behalf.
A simplified picture is:
application
│
│ "Where is example.com?"
▼
stub resolver
│
▼
recursive resolver
│
├── asks other DNS servers as required
│
▼
answer
│
▼
application
So there are already two perspectives hiding behind:
“my computer does a DNS lookup.”
The application asks for name information.
A resolver infrastructure does the work required to obtain it.
Recursive and iterative aren’t quite the same thing
This terminology confused me at first.
From the application’s perspective, I usually want something simple:
Please find the answer for me.
That is the idea behind a recursive query.
The resolver is being asked to return the answer, or report that it could not obtain one.
But while finding that answer, a resolver can make a series of iterative queries.
Those servers may respond:
I don't have the final answer,
but these servers are closer to it.
So a simplified lookup for:
www.example.com
might look conceptually like this:
CLIENT
│
│ recursive request
▼
RECURSIVE RESOLVER
│
│
├────► ROOT
│ │
│ └── "Try the .com servers"
│
├────► .COM SERVER
│ │
│ └── "Try the example.com servers"
│
├────► EXAMPLE.COM AUTHORITATIVE SERVER
│ │
│ └── "Here is the answer"
│
▼
CLIENT
That is simplified, but the distinction is useful:
recursive
│
└── "please obtain the answer for me"
iterative
│
└── "give me the best answer or referral you have"
The recursive resolver is doing the wandering so my application doesn’t have to.
Authoritative servers are different from recursive resolvers
Another distinction worth keeping straight:
A recursive resolver goes looking for answers.
An authoritative name server is responsible for answering from DNS data for the namespace it serves.
Conceptually:
recursive resolver
│
└── investigator
authoritative server
│
└── answers from authoritative
DNS data for the zones it serves
A single piece of DNS software might be capable of different roles, but the roles themselves are distinct.
That distinction seems likely to become important later.
DNS answers contain resource records
The information stored in DNS is represented using resource records.
A record has a type describing what sort of information it carries.
A few types immediately seem useful.
A
An A record contains an IPv4 address.
Conceptually:
www.example.com. A 192.0.2.10
So a query can essentially ask:
What IPv4 address information exists for this name?
AAAA
IPv6 uses AAAA records.
Conceptually:
www.example.com. AAAA 2001:db8::10
Apparently four As were deemed necessary.
Networking does occasionally reveal a sense of humour.
CNAME
A CNAME record says that one name is an alias for another canonical name.
Conceptually:
shop.example.com. CNAME web.example.com.
The resolver can then continue resolution using the canonical name.
So DNS can represent indirection between names, not just names and addresses.
NS
An NS record identifies a name server associated with the named domain.
This is part of how DNS distributes responsibility through its hierarchy.
Conceptually:
example.com. NS ns1.example.net.
MX
An MX record identifies a mail exchanger for a domain.
So:
example.com. MX mail.example.com.
can help mail systems determine where mail for the domain should be sent.
This makes the broader DNS model clearer:
domain name
│
├── A → IPv4 address
├── AAAA → IPv6 address
├── CNAME → canonical name
├── NS → name server
├── MX → mail exchanger
└── ...
DNS is therefore not just:
hostname → address
It is a distributed database of typed information associated with names.
That feels like a much better definition.
Caching stops every lookup becoming an expedition
There is an obvious problem with my resolver diagram.
If every request for:
example.com
required a fresh journey through the DNS hierarchy, that would create a great deal of unnecessary traffic.
DNS therefore supports caching.
Answers can have a TTL, or Time To Live, indicating how long cached information may be retained before it should be considered stale.
Conceptually:
first lookup
client
│
▼
resolver
│
├── root
├── .com
└── authoritative server
│
▼
answer
│
▼
cache
Then shortly afterwards:
second lookup
client
│
▼
resolver
│
├── cached answer still usable?
│
└── yes
│
▼
answer
This improves both performance and scalability.
It also introduces another interesting idea:
DNS changes do not necessarily become visible everywhere instantaneously.
If resolvers already hold valid cached information, they may continue using it until the relevant cache lifetime expires.
So DNS is distributed and cached.
That combination is starting to explain why DNS can occasionally feel slightly haunted after a configuration change.
DNS can cache failure too
Caching isn’t only for successful answers.
DNS also has mechanisms for negative caching.
That means information indicating that a name or requested data does not exist can itself be cached for a period.
So:
"there is no such name"
can be useful information worth remembering.
That seems small, but it prevents resolvers repeatedly performing the same unsuccessful work.
Success is data.
Failure can be data too.
My Bash exit-code note is apparently following me into networking.
What does “name does not exist” actually mean?
DNS responses have status information of their own.
One particularly useful result is commonly called:
NXDOMAIN
which indicates that the queried domain name does not exist in DNS.
That is different from:
the server did not respond
or from the case commonly called:
NODATA
where the name exists but there is no record of the type I asked for.
So, roughly:
NXDOMAIN
→ the queried name does not exist
NODATA
→ the name exists, but not the requested record type
Again, failure is more structured than:
DNS broke.
This seems to be another recurring systems lesson.
Good protocols distinguish different kinds of unsuccessful outcome.
DNS normally uses UDP, but that isn’t the whole story
After the previous TCP/UDP note, I wanted to know which transport DNS chooses.
The slightly irritating but educational answer is:
both.
Traditional DNS commonly uses UDP or TCP on port 53.
UDP is often used for ordinary queries.
If a response cannot be completed appropriately in the UDP exchange, the response can indicate truncation so the client can retry using TCP.
TCP is also used for operations such as zone transfers between DNS servers.
More importantly, full general-purpose DNS implementations are expected to support both transports.
So:
DNS
│
┌────┴────┐
│ │
UDP TCP
│ │
└────┬────┘
▼
IP
This is another good reminder that:
application protocol = transport protocol
is not a valid assumption.
The application protocol decides which transport services suit particular parts of its operation.
This is the traditional DNS transport model. Encrypted DNS transports also exist, but I am leaving those outside this first pass.
I can ask DNS questions myself
I don’t need a browser to trigger DNS.
Tools such as:
dig
let me send queries directly.
For example:
dig example.com A
asks for an A record.
Or:
dig example.com MX
asks for mail-exchanger information.
And:
dig example.com NS
asks for name-server records.
That makes the query model much more obvious:
name:
example.com
type:
A
question:
"What A records exist for example.com?"
rather than:
DNS, please internet this domain for me.
Progress.
dig +trace exposes the hierarchy
One experiment I particularly like is:
dig +trace example.com
Rather than simply giving me the final answer, this can make the delegation path through the DNS hierarchy much more visible.
dig +trace performs its own sequence of queries through that hierarchy rather than simply asking my configured recursive resolver for the final answer. So it is useful for exploring the delegation path, but it is not necessarily a transcript of what my ordinary resolver happened to do.
Conceptually, I can watch the lookup move through:
root
│
▼
top-level domain
│
▼
authoritative infrastructure
│
▼
answer
That turns the DNS hierarchy from a diagram into observable behaviour.
I suspect dig is going to become one of those tools where:
I'll just check one thing
quietly consumes an afternoon.
DNS does not create the network connection
This distinction is worth making explicit after looking at HTTP.
Suppose I type:
https://example.com/
DNS does not fetch the webpage.
It doesn’t establish the HTTP conversation.
Its part of the journey is narrower.
Conceptually:
example.com
│
▼
DNS
│
└── address information
│
▼
TCP connection
│
▼
TLS
│
▼
HTTP
DNS helps one layer discover information required before communication can continue.
Then other protocols take over.
Again, the Internet is looking less like one system and more like a stack of cooperating systems.
The mental model I’m keeping
The old model:
example.com
│
▼
DNS
│
▼
1.2.3.4
The new model:
APPLICATION
│
│ asks about a name
▼
STUB RESOLVER
│
▼
RECURSIVE RESOLVER
│
cache available?
│ │
yes no
│ │
│ ▼
│ ROOT
│ │
│ ▼
│ TLD
│ │
│ ▼
│ AUTHORITATIVE
│ NAME SERVER
│ │
└────┬─────┘
▼
RESOURCE RECORDS
│
┌──────────┼──────────┐
│ │ │
▼ ▼ ▼
A CNAME MX
address alias mail
At its heart, DNS still starts with an extremely simple question:
What information do you have about this name?
The cleverness is in how a global system can answer that question without one server knowing everything.
Hierarchy distributes responsibility.
Delegation lets different organisations control different parts of the namespace.
Resolvers do the searching.
Authoritative servers provide answers for the data they serve.
Caching prevents every lookup becoming a world tour.
And resource-record types let the question be more specific than simply:
What’s the IP?
That is a considerably more interesting system than the phone book I thought I was using.
References and further reading
DNS concepts and architecture
RFC 1034: Domain Names — Concepts and Facilities The foundational description of DNS concepts, including the hierarchical domain namespace, name servers, resolvers, zones, caching and recursive and iterative query behaviour.
DNS protocol and resource records
RFC 1035: Domain Names — Implementation and Specification
Defines the DNS protocol message format and many of the resource-record types discussed here, including A, NS, CNAME, MX and PTR. It also describes DNS communication over UDP and TCP.
IPv6 DNS records
RFC 3596: DNS Extensions to Support IP Version 6
Defines the AAAA resource record used to store IPv6 addresses in DNS.
Negative caching
RFC 2308: Negative Caching of DNS Queries Defines negative caching behaviour for DNS, including caching information about names or requested records that do not exist.
NXDOMAIN behaviour
RFC 8020: NXDOMAIN: There Really Is Nothing Underneath
Clarifies the meaning of an NXDOMAIN response and its implications for names beneath a non-existent DNS node.
DNS transport over TCP
RFC 7766: DNS Transport over TCP — Implementation Requirements Clarifies modern requirements for DNS over TCP, including the expectation that general-purpose DNS implementations support both UDP and TCP.
Linux DNS query tool
BIND 9 dig documentation
Documentation for dig, the DNS query tool used in the examples above to inspect records and resolution behaviour.