I have spent the last few months looking increasingly closely at what happens inside one Linux machine.
Programs become processes. Processes have state. They inherit file descriptors and environments. They make requests to the kernel.
Eventually, though, one process wants to talk to something that is not on the same machine.
That introduces a new question:
How does some data created by one program actually reach another computer?
My old mental model was approximately:
my computer
│
│ internet
▼
other computer
Technically difficult to dispute.
Also almost completely useless.
So I want to follow a piece of data across a network and see what has to happen between those two boxes.
For this first pass I am deliberately concentrating on an ordinary IPv4 network using Ethernet.
There are many details I am leaving for later.
Starting with two processes
Imagine a program on my machine wants to send:
Hello
to a server.
At the application level, that may be all either program really cares about:
client process
"Hello"
│
▼
server process
The application does not normally need to know how an Ethernet frame is constructed or which routers exist between the two machines.
That is the first useful idea.
Networking is layered.
The Internet protocol suite is traditionally described using layers including:
application
transport
internet
link
Each layer solves a different part of the communication problem.
So our innocent:
Hello
is about to acquire quite a wardrobe.
Layer 1: the application has some data
Suppose an application creates five bytes:
Hello
Those bytes belong to whatever application protocol the two programs have agreed to use.
That could eventually be:
HTTP
DNS
SSH
SMTP
or an application-specific protocol of my own.
For now, I do not actually care what those five bytes mean.
From the lower layers’ perspective they are simply data that needs to be transported.
So we begin with:
┌───────────────┐
│ Hello │
└───────────────┘
application data
The next question is:
Which communicating endpoint on the destination machine should receive it?
An IP address alone does not answer that.
A computer may simultaneously be running:
a web server
an SSH server
a database
a DNS service
and dozens of other networked processes
Something else has to help distinguish between the communicating endpoints involved.
That is where the transport layer enters.
Layer 2: transport
Two transport protocols appear constantly in Internet networking:
TCP and UDP.
They offer applications different communication models.
TCP provides applications with a reliable, ordered byte-stream abstraction between endpoints.
UDP offers a much smaller datagram-oriented mechanism without guaranteeing delivery or duplicate protection.
I don’t need to understand all of TCP yet.
The thing I need from the transport layer today is the idea of ports.
A transport endpoint can include a port number identifying the application or service involved.
Conceptually:
destination host
192.0.2.20
│
├── port 22 SSH
├── port 53 perhaps DNS
├── port 443 perhaps HTTPS
└── port 9000 perhaps my application
So my application isn’t merely communicating with:
192.0.2.20
It may be communicating with something closer to:
192.0.2.20 : 9000
The IP address helps identify the destination host or interface.
The transport-layer port helps identify the communicating endpoint/service on that host.
Our data can now be thought of as being wrapped in transport information:
┌───────────────────────────────┐
│ transport header │
├───────────────────────────────┤
│ Hello │
└───────────────────────────────┘
For TCP this unit is commonly called a segment.
For UDP it is a datagram.
Already, the original application data is no longer travelling by itself.
A quick TCP versus UDP mental model
For now I am keeping this deliberately simple.
TCP
Think:
connection-oriented
ordered
reliable
byte stream
TCP tracks sequence information and contains mechanisms intended to provide reliable, in-order delivery to applications.
UDP
Think:
datagram-oriented
small protocol mechanism
no delivery guarantee from UDP itself
no ordering guarantee from UDP itself
This does not mean:
TCP = good
UDP = bad
It means the application is choosing different semantics.
A DNS query, voice stream and file transfer may not all want exactly the same transport behaviour.
That deserves its own investigation later.
Today I just need the transport layer to hand something down to IP.
Layer 3: IP needs to get it to another host
The transport data is handed to the Internet Protocol.
IP’s job is different.
It is concerned with moving datagrams between addressed hosts across interconnected networks.
For IPv4, an IP packet contains a header carrying information including a source address and destination address.
Our packet can now be pictured as:
┌───────────────────────────────┐
│ IPv4 header │
│ │
│ src: 192.0.2.10 │
│ dst: 198.51.100.20 │
├───────────────────────────────┤
│ transport header │
├───────────────────────────────┤
│ Hello │
└───────────────────────────────┘
This layering is important.
The IP layer does not suddenly replace TCP.
Instead:
application data
│
▼
transport data
│
▼
IP datagram
Each layer wraps information needed for its own job around the data handed down from above.
This is commonly called encapsulation.
Addresses and routes are not the same thing
One sentence from the original IPv4 specification helped my mental model enormously.
It distinguishes between three concepts:
name
address
route
Roughly:
name → what am I looking for?
address → where is it?
route → how do I get there?
Knowing a destination IP address therefore does not by itself tell my machine exactly where to send the next Ethernet frame.
The host has to determine the appropriate route.
The routing table
On Linux I can inspect routes with:
ip route
A simplified table might contain something conceptually like:
192.168.1.0/24 dev eth0
default via 192.168.1.1 dev eth0
This tells the system something important.
A destination within:
192.168.1.0/24
can be reached directly through the local network.
Something elsewhere should be sent towards:
192.168.1.1
which might be my default gateway.
So before sending an IP packet, the host effectively asks:
Where should this packet go next?
Not:
What is every router between London and California?
Just:
What is my next hop?
That is a much more manageable problem.
Local destination or remote destination?
Suppose my machine is:
192.168.1.10/24
and I want to send to:
192.168.1.20
Those addresses belong to the same /24 network.
My host can try to deliver the packet directly across the local link.
But suppose the destination is:
198.51.100.20
That is not part of the local network.
My computer instead uses its routing information and may decide:
send this towards the default gateway
Notice something subtle.
Ignoring things such as NAT, the IP destination remains the final destination:
198.51.100.20
But the next machine that physically receives the frame may only be:
my router
This was one of the first networking distinctions that really clicked for me:
The final IP destination and the next-hop link-layer destination are not necessarily the same machine.
To understand why, I need one more kind of address.
Ethernet needs a hardware address
On an Ethernet network, delivery across the local link uses Ethernet addresses, commonly called MAC addresses.
So my host may know:
next-hop IP:
192.168.1.1
but still need to answer:
Which Ethernet address corresponds to 192.168.1.1?
For IPv4-over-Ethernet networks, that is one of the jobs of ARP, the Address Resolution Protocol.
ARP exists because an Internet Protocol address and an Ethernet hardware address are different kinds of identifiers.
Conceptually:
IP address
192.168.1.1
│
│ ARP
▼
Ethernet address
aa:bb:cc:dd:ee:ff
If the mapping is not already known, the host can send an ARP request over the local network asking which device owns the target protocol address.
The relevant device replies with its hardware address.
The host can then construct the Ethernet frame required for the actual IP traffic.
The frame appears
Now our original five bytes have accumulated another layer.
Conceptually:
┌───────────────────────────────────────┐
│ Ethernet header │
│ │
│ source MAC │
│ destination MAC │
├───────────────────────────────────────┤
│ IPv4 header │
│ │
│ source IP │
│ destination IP │
├───────────────────────────────────────┤
│ transport header │
│ │
│ source port │
│ destination port │
├───────────────────────────────────────┤
│ Hello │
└───────────────────────────────────────┘
This is the first point where the word packet starts becoming annoyingly imprecise.
Depending on the layer I am discussing, I may mean:
application data
transport segment/datagram
IP packet/datagram
Ethernet frame
I suspect being more precise about the layer will save me confusion later.
What actually goes onto the network?
The network interface now has a frame that can be transmitted across the local medium.
At the physical level, the frame ultimately becomes signals.
Depending on the network, those may involve:
electrical signalling
light
radio
The exact physical encoding is another entire subject.
For my current mental model, it is enough to say:
structured bits
│
▼
network interface
│
▼
physical signalling
│
▼
local network
Our Hello has finally left the machine.
Sort of.
It is buried several layers deep inside a frame.
The router receives the frame
Suppose the destination is outside my local network.
The Ethernet frame is addressed to my router’s local interface.
The router receives that frame.
Here is the important bit:
the Ethernet frame was only for this local hop.
The router does not simply forward the exact same Ethernet frame across the Internet untouched.
Instead, conceptually it:
receives local frame
│
▼
extracts IP packet
│
▼
examines destination IP
│
▼
consults routing information
│
▼
chooses next hop/interface
│
▼
updates forwarding-related
IP information as required
│
▼
encapsulates the packet inside
framing for the next link
│
▼
sends it onwards
So:
Ethernet frame A
│
▼
Router
│
▼
link-layer frame B
The link-layer envelope can change from hop to hop.
The IP datagram continues towards its destination too, although a router may update IP-layer fields as part of forwarding. For example, an IPv4 router reduces the packet’s TTL as it forwards it.
IP provides the addressing that allows the traffic to move across multiple interconnected networks towards its destination.
This suddenly makes the layers feel much less theoretical.
They have different geographic scopes.
Roughly:
Ethernet
local hop
IP
across interconnected networks
TCP/UDP
end-to-end application transport
application protocol
meaning of the conversation
Routers repeat the decision
Our packet may travel through several routers.
Something like:
Host A
│
▼
Router 1
│
▼
Router 2
│
▼
Router 3
│
▼
Host B
Each router does not need a complete philosophical understanding of:
Why is Miri sending "Hello"?
It primarily needs enough forwarding information to decide:
Where should this packet go next?
This hop-by-hop forwarding is what lets very large networks be assembled from smaller networks.
The Internet is not one enormous Ethernet cable.
It is a network of networks.
That suddenly makes the name Internet Protocol rather literal.
Eventually the destination network is reached
At some point, a router reaches a network on which the destination host is directly reachable.
The final link still needs local delivery.
On our IPv4/Ethernet example, the router may use ARP to learn the destination host’s Ethernet address if it does not already know it.
It can then construct a frame whose:
destination MAC
belongs to the target host.
The destination machine receives the frame.
Now the process begins travelling up through the layers.
Decapsulation
The receiving host can work through the nested headers in the opposite direction:
Ethernet frame
│
▼
remove/process Ethernet information
│
▼
IP packet
│
▼
process IP information
│
▼
TCP segment / UDP datagram
│
▼
identify destination endpoint
│
▼
application data
│
▼
server process
Our original:
Hello
has survived its trip through several layers of protocol machinery and arrived somewhere an application can make sense of it again.
This process is often called decapsulation.
So the overall pattern is beautifully symmetrical:
SENDING HOST
application
│
│ data
▼
transport
│
│ transport header + data
▼
IP
│
│ IP header + transport data
▼
link
│
│ frame
▼
=========== NETWORK ===========
▼
link
│
▼
IP
│
▼
transport
│
▼
application
RECEIVING HOST
Where does DNS fit?
There is one problem with everything I have described so far.
Humans generally don’t begin with:
198.51.100.20
We begin with names:
example.com
Before an application can communicate with a remote service by name, it may therefore need to resolve that name into information that includes an IP address.
That is where the Domain Name System, DNS, enters.
Very roughly:
example.com
│
│ DNS resolution
▼
IP address
│
▼
routing + transport
DNS itself is a networked protocol, which means there is something wonderfully recursive happening here.
To learn where another network service lives, my machine may first have to communicate with a DNS service over the network.
I am resisting that rabbit hole today.
It can have its own article.
Local and remote traffic use the same layers differently
The local-versus-remote decision now feels much clearer.
Routing determines whether the destination is directly reachable or whether the packet should go to a next-hop router. The link layer then handles delivery across that particular local hop.
Before this, I had mentally blurred IP routing and Ethernet delivery into one thing.
They are cooperating layers.
A useful Linux experiment
Linux exposes enough information to start observing some of this.
My addresses
ip addr
My routes
ip route
My neighbours
ip neigh
The neighbour table can show mappings learned for nearby hosts or routers, including the relationship between an IP address and a link-layer address.
I can then try:
ping -c 1 <local-ip-address>
and inspect:
ip neigh
again.
There is something satisfying about seeing a protocol concept become an entry in the operating system’s actual networking state.
Later I want to look at packets themselves with something such as tcpdump.
That seems both useful and potentially dangerous to my remaining free time.
What about switches?
There is another device I have largely skipped over.
An Ethernet network commonly contains switches.
At a simplified level, a switch forwards Ethernet frames within a local network using link-layer addressing.
A router, meanwhile, forwards traffic between IP networks.
So another early mental distinction is:
switch
│
└── thinks primarily about local link-layer forwarding
router
│
└── thinks primarily about forwarding between IP networks
Real networks naturally become more sophisticated than that sentence, but it is already considerably better than calling every box with blinking lights “the router”.
Ports, IP addresses and MAC addresses solve different problems
I used to see these as three varieties of “network address”.
That hides why all three exist.
I now think of them roughly like this:
MAC address
│
└── where on this local link?
IP address
│
└── which networked destination?
port
│
└── which transport endpoint/application?
Or:
┌────────────────────────────────────────┐
│ Application │
│ │
│ "What does the message mean?" │
├────────────────────────────────────────┤
│ Transport │
│ │
│ "Which communicating endpoint?" │
│ ports │
├────────────────────────────────────────┤
│ Internet │
│ │
│ "Which host/network destination?" │
│ IP addresses │
├────────────────────────────────────────┤
│ Link │
│ │
│ "Where does this frame go locally?" │
│ MAC addresses │
└────────────────────────────────────────┘
The layers are not bureaucracy.
They are separation of concerns.
Putting the journey together
Suppose Host A at:
192.168.1.10
wants to send Hello to Host B at:
198.51.100.20
port 9000
and Host A’s router is:
192.168.1.1
The journey now looks roughly like this:
"Hello"
│
▼
transport adds endpoint information
│
▼
IP adds source and destination addresses
│
▼
routing chooses the next hop
│
▼
ARP may resolve that next hop to a local MAC address
│
▼
link-layer framing carries the IP packet across one hop
│
▼
routers repeat the forwarding decision
│
▼
destination host
│
▼
IP → transport → port 9000 → server process
│
▼
"Hello"
The important part is that each layer solves a different problem. The application does not need to know the route, and an intermediate router does not need to understand what Hello means.
The mental model I’m keeping
Data does not simply:
travel across the Internet.
It repeatedly moves through layers of abstraction.
The application creates meaningful data.
The transport layer provides communication between application endpoints.
IP gives the traffic source and destination addressing across interconnected networks.
Routing determines how to move the packet towards that destination.
The local link layer gets it across one particular network hop.
Routers repeat the forwarding process until the destination network is reached.
Then the receiving host unwraps the layers until the original application data reaches the intended process.
So:
application
│
▼
transport
│
▼
IP
│
▼
link
│
▼
physical network
...
physical network
│
▼
link
│
▼
IP
│
▼
transport
│
▼
application
The Internet suddenly looks less like a cloud.
It looks like a series of agreements.
Each layer knows enough to perform one part of the journey and hand the result to the next.
That seems like a much better place from which to start learning networking.
References and further reading
The following are the primary technical references behind the concepts used in this article.
Internet architecture and protocol layering
RFC 1122: Requirements for Internet Hosts — Communication Layers A foundational description of the Internet host architecture, including the link, IP and transport layers used throughout this article.
IPv4
RFC 791: Internet Protocol The original IPv4 specification. It defines the IPv4 datagram format and describes the role of IP in moving datagrams across interconnected networks.
TCP
RFC 9293: Transmission Control Protocol (TCP) The current consolidated TCP specification. Useful for TCP connections, ports, sequence numbers, reliable delivery and the byte-stream abstraction.
UDP
RFC 768: User Datagram Protocol The original UDP specification. Its brevity is useful in itself: UDP deliberately provides a small datagram-oriented transport mechanism without TCP’s reliability machinery.
Address Resolution Protocol
RFC 826: An Ethernet Address Resolution Protocol Defines ARP, including the mapping between protocol addresses such as IPv4 addresses and Ethernet hardware addresses on a local network.
DNS
RFC 1034: Domain Names — Concepts and Facilities Introduces the architecture and conceptual model of the Domain Name System.
RFC 1035: Domain Names — Implementation and Specification Defines DNS message formats, resource records and protocol behaviour in greater detail.
Linux networking tools
Linux ip-route(8) manual page
Documents the ip route commands used to inspect and manipulate Linux routing tables.
Linux ip-neighbour(8) manual page
Documents Linux neighbour tables, including the relationship between protocol addresses and link-layer addresses used by ARP.
Linux ip-address(8) manual page
Documents the ip addr interface used to inspect IP addresses assigned to Linux network interfaces.
A note on terminology
The vocabulary in networking can become slippery because the generic word packet is often used for several different protocol data units.
In this article I have tried to distinguish them where useful:
application layer → application data
TCP → segment
UDP → datagram
IP → packet / datagram
Ethernet → frame
The exact terminology varies somewhat between documentation and everyday engineering conversation, so the important thing is usually to make clear which layer is being discussed.