Caching: trading freshness for speed

Suppose a customer opens a product page.

The application needs to retrieve:

• product name
• description
• price
• images
• category
• publication status

The straightforward design is:

CUSTOMER
    │
    ▼
APPLICATION
    │
    ▼
DATABASE
    │
    ▼
APPLICATION
    │
    ▼
CUSTOMER

Every request reaches the authoritative data store.

That has an attractive property:

The application reads the current committed value from the system that owns it.

It can also become expensive.

The same popular product may be requested thousands of times even though its description changes once a month.

Each request might require:

• a network round trip
• a database connection
• query parsing
• index traversal
• row access
• data decoding
• response construction

The database repeats work whose result is often unchanged.

A cache offers another path:

CUSTOMER
    │
    ▼
APPLICATION
    │
    ▼
CACHE
  │    │
 hit   miss
  │    │
  │    ▼
  │  DATABASE
  │    │
  └────┘
    │
    ▼
CUSTOMER

On a cache hit, the application can avoid the more expensive data-source operation.

The response may arrive faster.

The database may receive less load.

Everything appears improved.

Except the cache now holds a copy.

The authoritative product price might be:

£249.00

while the cached value still says:

£229.00

The performance improvement has created a consistency question.

That seems to be the central trade:

Caching improves access by serving a nearby or cheaper copy, but that copy can become older than its source.

The difficult part is not putting data into a cache.

The difficult part is deciding what the copy means once the source changes.

1. A cache is a managed copy

Why cache at all?

A cache stores data that can, in principle, be obtained or reconstructed elsewhere.

Examples include:

• database query results
• HTTP responses
• rendered pages
• compiled files
• DNS answers
• session information, when it can be safely reconstructed or another authoritative representation exists
• product summaries
• authentication metadata

The cache exists because retrieving or calculating the original value has a cost.

That cost might be:

• latency
• CPU time
• database work
• network traffic
• storage operations
• external API charges

A useful first model is:

AUTHORITATIVE SOURCE
        │
        │ copy
        ▼
       CACHE
        │
        │ cheaper access
        ▼
     CONSUMER

The source owns the truth.

The cache owns a disposable representation of that truth.

But “disposable” does not mean unimportant.

If an application serves the cached value to a customer, that value becomes part of the application’s behaviour.

A wrong cached price is still a wrong price.

A stale permission decision is still a security problem.

A missing cache entry can still create a production incident when every request stampedes towards the database.

The copy may be replaceable.

Its behaviour still requires design.

Without a cache:

request
   │
   ▼
read authoritative source
   │
   ▼
return result

With a cache:

request
   │
   ▼
look up cache key
   │
   ├── hit
   │     │
   │     └── return cached value
   │
   └── miss
         │
         ▼
      read source
         │
         ▼
      populate cache
         │
         ▼
      return value

The cache introduces new decisions:

• How is the key constructed?
• How long may the value remain?
• Who populates it?
• Who invalidates it?
• What happens when it is unavailable?
• What happens when the source changes?
• What happens when many requests miss together?
• Can a missing result be cached?
• How do I know whether the cache is helping?

The read became faster.

The architecture became larger.

A cache hit occurs when the requested entry is present and usable.

request for product:chair-42
            │
            ▼
          CACHE
            │
            ├── entry found
            └── return product data

A cache miss occurs when the entry is not available.

Reasons can include:

• the key was never cached
• the entry expired
• the entry was evicted
• the entry was invalidated
• the cache restarted
• the key was constructed incorrectly

The application then retrieves the value from its source.

request
   │
   ▼
cache miss
   │
   ▼
database query
   │
   ▼
cache population
   │
   ▼
response

The proportion of requests served successfully by the cache is often described as the hit ratio.

A high hit ratio can indicate that the cache is avoiding a substantial amount of source work.

It does not by itself prove that the cache is correct or worthwhile.

A cache could achieve an impressive hit ratio by serving values long after they should have changed.

Performance measurements need freshness and correctness context.

Caches at different distances

“Caching makes things faster” needs a comparison.

An in-process memory lookup may be faster than a network request to a remote cache.

A remote cache may be faster than a relational database query.

A database index lookup may already be fast enough.

The path could look like:

PROCESS MEMORY
      │
      │ fastest and closest
      ▼
REMOTE CACHE
      │
      ▼
DATABASE
      │
      ▼
EXTERNAL SERVICE
      │
      │ potentially slowest and furthest away
      ▼

Each layer has different costs.

In-process cache

application process
│
└── local memory cache

Advantages:

• no network request
• very low access latency
• simple access path

Costs:

• each application instance has its own copy
• memory is limited
• entries disappear on restart
• invalidation must reach several instances

Shared remote cache

application instances
        │
        ▼
   shared cache

Advantages:

• one logically shared caching layer
• larger central capacity
• independent cache lifecycle

Costs:

• network latency
• another service to operate
• shared failure point
• connection and capacity limits

Client and intermediary caches

HTTP responses may be cached by:

• browsers
• forward proxies
• reverse proxies
• content delivery networks

These copies can move content closer to users and reduce application traffic.

They also place cached state outside the application’s immediate process.

The cache-control contract becomes especially important because the application may not be able to delete every distributed copy directly.

Authority and reconstruction

Suppose catalog owns the current product price.

CATALOG DATABASE

product chair-42
price £249.00

A cache contains:

CACHE

product chair-42
price £229.00

The cache is not a second authority allowed to disagree indefinitely.

It is a projection that has fallen behind its owner.

AUTHORITATIVE FACT

catalog price = £249.00


DERIVED COPY

cached price = £229.00

This distinction matters when recovering from cache loss.

If the cache disappears, the system should normally be able to reconstruct its entries from the authoritative source.

cache lost
    │
    ▼
read source
    │
    ▼
rebuild cache

If losing the cache destroys unique business data, it was not merely a cache.

It had quietly become a system of record while everybody continued calling it disposable.

2. Freshness, expiration, and invalidation

Freshness is a business requirement

Not all data needs the same freshness.

A product description might tolerate being several minutes old.

A stock quantity may require a much smaller window.

A permission revocation may need to take effect almost immediately.

A daily sales report may intentionally represent data as of a completed reporting period.

So I should not ask only:

How long should the time to live be?

I should first ask:

How stale may this information become before the system’s behaviour is unacceptable?

Examples:

product description
    acceptable staleness: perhaps minutes


search suggestion
    acceptable staleness: perhaps minutes or hours


inventory availability
    acceptable staleness: perhaps seconds,
    depending on how it is used


account suspension
    acceptable staleness: potentially very small


country reference data
    acceptable staleness: potentially days

The correct answer comes from the meaning of the data.

There is no universal professional-looking cache duration such as five minutes.

Five minutes can be conservative for one value and catastrophic for another.

TTL balances freshness and load

A cache entry can be assigned a time to live, or TTL.

For example:

key:
    product:chair-42

value:
    product summary

TTL:
    300 seconds

After five minutes, the entry expires.

The next request misses and retrieves a newer value.

entry stored
    │
    ▼
usable for 300 seconds
    │
    ▼
expires
    │
    ▼
next request reloads source

This gives the stale value a maximum intended lifetime in the cache.

But the exact freshness experienced by a customer depends on timing.

Suppose the source changes one second after the cache is populated.

• 12:00:00 cache populated
• 12:00:01 source changes
• 12:05:00 cache expires

The cache may serve the old value for almost the full five minutes.

If the source changes one second before expiry, the stale period is much smaller.

TTL provides a bounded approximation.

It does not keep the cache continuously synchronised.

A shorter TTL usually means:

• fresher values
• more cache misses
• more source traffic

A longer TTL usually means:

• older values may persist
• fewer cache misses
• less source traffic

Conceptually:

SHORT TTL

freshness ↑
source load ↑
cache hit opportunity ↓


LONG TTL

freshness ↓
source load ↓
cache hit opportunity ↑

This is why TTL is both a consistency choice and a capacity choice.

Reducing a TTL may improve freshness while accidentally overwhelming the database.

Increasing it may protect the database while serving unacceptable state.

The number should be connected to:

• allowed staleness
• update frequency
• request frequency
• source capacity
• failure behaviour

Expiration, invalidation, and races

These words are related but different.

Expiration

The entry becomes unusable because its configured lifetime ends.

entry age reaches TTL
        │
        ▼
entry expires

Invalidation

The system deliberately removes or supersedes an entry because the source changed or the entry is no longer trustworthy.

source changes
      │
      ▼
invalidate related key

A cache can rely only on expiration:

source changed
      │
      ▼
wait for TTL
      │
      ▼
cache refreshes later

Or it can actively invalidate:

source changed
      │
      ▼
delete cache entry
      │
      ▼
next read reloads source

Active invalidation can reduce stale time.

It also creates another distributed operation that can fail.

Suppose catalog updates a product:

price £229.00 → £249.00

The application then deletes:

product:chair-42

from the cache.

The sequence might be:

update database
      │
      ▼
commit
      │
      ▼
delete cache key

What happens if the application crashes after the commit but before deleting the cache entry?

DATABASE
    │
    └── £249.00


CACHE
    │
    └── £229.00

The database transaction succeeded.

The invalidation did not.

The cache may remain stale until TTL expiry or another repair mechanism runs.

That makes invalidation a distributed consistency problem.

The famous difficulty is not discovering how to execute:

DELETE key

The difficulty is ensuring that all relevant copies stop being trusted after the authoritative state changes.

Perhaps I reverse the operations:

delete cache entry
      │
      ▼
update database

That creates another race.

WRITER                       READER

delete cache entry

                             cache miss
                             read old database value
                             repopulate old value

commit new database value

The cache again contains stale data.

No ordering of two independent operations automatically makes them one atomic change.

There is also a late cache-fill race:

READER                         WRITER

cache miss
read old database value

                               update database
                               commit
                               delete cache key

populate cache with
the previously read old value

The writer invalidated the key, but the reader repopulated it afterwards with a value read before the commit. Versions, coordinated loaders, change streams and bounded TTLs can reduce this risk, but each adds its own contract.

This is one reason TTL often remains useful even when active invalidation exists. Expiration provides an eventual escape route when invalidation is delayed or lost.

Serving stale data deliberately

Suppose catalog is temporarily unavailable.

The cache contains product descriptions that are three minutes old.

The application has several choices:

• fail every product request
• serve stale product descriptions
• serve stale descriptions but suppress actions that require current data

Serving stale data can improve availability for information where age is tolerable.

For example, it may be acceptable to show:

• product name
• description
• images

from an older cache entry.

It may be less acceptable to confirm:

• current price
• stock reservation
• account permission

from the same stale copy.

This suggests that one cached object can be too coarse.

Different fields may have different freshness requirements.

PRODUCT PRESENTATION
    │
    └── slower-changing, longer cache lifetime


PURCHASING DECISION
    │
    └── current validation from authoritative owner

Caching should follow the semantics of the operation, not merely the convenience of one large response object.

3. Common caching patterns

Pattern comparison

Caching patterns differ mainly in who loads values, who coordinates writes, and where acknowledged state lives.

The names do not remove the underlying questions. Each pattern still needs a freshness, failure and authority policy.

Cache-aside and read-through

A common pattern is cache-aside, also called lazy loading.

On a read:

• 1. look in cache
• 1. return value on hit
• 1. read database on miss
• 1. populate cache
• 1. return value

In simplified pseudocode, ignoring concurrency, timeout and cache-failure handling:

func GetProduct(ctx context.Context, id string) (Product, error) {
    key := "product:" + id

    if product, ok := cache.Get(key); ok {
        return product, nil
    }

    product, err := repository.GetProduct(ctx, id)
    if err != nil {
        return Product{}, err
    }

    cache.Set(key, product, fiveMinutes)

    return product, nil
}

On a write:

• 1. update authoritative source
• 1. invalidate cached copy

The cache only contains values that have been requested.

That avoids preloading every possible record.

The application, however, becomes responsible for:

• miss handling
• population
• expiration
• invalidation
• error handling

The cache is beside the application data path rather than transparently inside the database.

In a read-through model, the application asks the cache for data and the caching layer retrieves missing values through a configured loader.

APPLICATION
    │
    ▼
CACHE
  │    │
 hit   miss
  │    │
  │    ▼
  │  loader
  │    │
  │    ▼
  │  source
  └────┘

The application sees one read interface.

This can centralise loading behaviour.

It also makes the caching layer more closely coupled to the source or loader implementation.

The broad question remains unchanged:

What happens when the cached copy is absent or stale?

The pattern merely assigns that responsibility to a different component.

Write-through and write-behind

In a write-through design, writes pass through the caching layer.

APPLICATION
    │
    ▼
CACHE LAYER
    │
    ├── update source
    └── update cache

This can keep cached entries current after successful writes.

But “together” still needs careful interpretation.

If the cache and database are independent systems, one can succeed while the other fails.

• database updated
• cache update failed

or:

• cache updated
• database update failed

The write path needs a consistency policy.

A cache update should not become the acknowledged business result before the authoritative write is secure, unless the cache itself is deliberately part of the authoritative storage design.

A write-behind, or write-back, design updates the cache first and persists the change to the source later.

APPLICATION
    │
    ▼
CACHE
    │
    │ acknowledge
    ▼
APPLICATION CONTINUES

later:

CACHE
    │
    ▼
DATABASE

This can improve apparent write latency and combine several updates.

It also gives the cache a much larger responsibility.

If the cache loses an acknowledged write before it reaches the database, business data may be lost.

The design now needs:

• durable queued writes
• ordering
• retry handling
• failure recovery
• backpressure
• conflict policy

This is no longer merely a disposable read optimisation.

Write-behind should not be selected because its diagram contains fewer waiting arrows.

It changes where acknowledged state lives.

Refresh-ahead

A cache can refresh an entry before it expires.

entry nearing expiry
        │
        ▼
background refresh
        │
        ▼
replace cached value

This can help prevent frequently requested values from falling out of cache and causing synchronous misses.

It is useful when:

• requests are predictable
• source refresh is relatively expensive
• slightly stale values remain acceptable

It also creates background work and requires a policy for refresh failure.

Should the cache:

• serve the older value temporarily?
• remove it immediately?
• retry in the background?
• force the next caller to reload it?

The correct answer depends on how dangerous staleness is compared with unavailability.

4. HTTP caching

Freshness and validation

HTTP includes caching semantics so that servers, browsers and intermediaries can communicate reuse rules.

A response might contain:

HTTP/1.1 200 OK
Cache-Control: public, max-age=300
ETag: "product-chair-42-v17"
Content-Type: application/json

max-age=300 says the response can be considered fresh for 300 seconds under the applicable cache rules.

The entity tag gives the representation a validator:

"product-chair-42-v17"

After the response becomes stale, a client can make a conditional request:

GET /products/chair-42 HTTP/1.1
Host: shop.example
If-None-Match: "product-chair-42-v17"

If the representation has not changed:

HTTP/1.1 304 Not Modified
ETag: "product-chair-42-v17"

The client can reuse its stored representation without downloading the full body again.

This is still caching.

The cache is validating an older copy before reuse.

An HTTP cache can treat a response as:

fresh
    │
    └── reusable without contacting origin,
        subject to request and response rules


stale
    │
    └── may require validation

Stale does not necessarily mean incorrect.

It means the response has passed its defined freshness lifetime.

A stale response may still match the origin exactly.

Validation answers:

Has this representation changed?

cached representation
        │
        ▼
send validator
        │
        ├── unchanged
        │      └── reuse body
        │
        └── changed
               └── retrieve new body

This avoids transferring a full response when the old copy remains valid.

It also requires the origin to generate reliable validators.

Cache-control directives and cache scope

HTTP cache directives contain some famously unhelpful naming.

A response with:

Cache-Control: no-cache

may still be stored.

The directive requires successful validation with the origin before the stored response is reused.

By contrast:

Cache-Control: no-store

instructs caches not to store the response under the protocol’s rules.

Conceptually:

no-cache
    │
    └── storage may occur,
        reuse requires validation


no-store
    │
    └── do not store

This is a useful reminder that familiar English words are not substitutes for reading the protocol contract.

An HTTP response can contain user-specific information.

For example:

• customer name
• basket contents
• account balance

A browser cache belongs to one user context.

A shared proxy or content delivery network may serve many users.

A response intended only for a private cache can use:

Cache-Control: private

A response that may be stored by shared caches can use:

Cache-Control: public

public permits storage by shared caches, subject to the other caching rules that apply.

The response may also vary according to request headers:

Vary: Accept-Encoding

The cache then includes the relevant request-header values when selecting a stored response.

If the cache key omits a dimension that changes the response, one user or representation can receive another’s data.

Cache-key design is therefore part of correctness and security.

5. Identity and cached representations

Keys define representation identity

Suppose a product response differs by:

• product ID
• currency
• language
• customer segment

This key is insufficient:

product:chair-42

A request for GBP pricing and en-GB content might populate a representation that another request should not receive.

A more complete key could include the dimensions that define the cached value:

product:chair-42:currency:GBP:language:en-GB

The exact format matters less than the rule:

Two requests that can legitimately produce different values must not accidentally share one cache entry.

Missing dimensions can leak:

• the wrong locale
• the wrong currency
• the wrong permissions
• another user’s personalised result

Adding every possible dimension, however, can reduce reuse and increase cache size.

The key needs to encode the true identity of the representation.

Suppose the system caches:

user 72 may view order 5001

A key such as:

order-access:5001

does not include the user.

The first permitted request may populate:

allowed = true

A later request from a different user could reuse that decision.

The correct key may need:

order-access:user-72:order-5001

It may also need to reflect:

• role
• tenant
• policy version
• resource state

Security-sensitive caches deserve particularly conservative freshness and key policies.

A fast wrong answer from an authorisation cache is still an access-control failure.

Negative caching

Suppose a client repeatedly requests a product that does not exist:

product:missing-chair

Without negative caching:

request
   │
   ▼
cache miss
   │
   ▼
database query
   │
   ▼
not found

The same unsuccessful query reaches the database every time.

A cache can temporarily store:

product missing

This is negative caching.

key:
    product:missing-chair

value:
    not found

TTL:
    30 seconds

It can reduce repeated work for absent values.

But the absence can become stale too.

If the product is created during those 30 seconds, the cache may continue reporting that it does not exist.

Negative results often deserve shorter lifetimes because absence can change.

Errors require even more care.

Caching a temporary database outage as though the product does not exist would convert an infrastructure failure into false business information.

The cache must distinguish:

authoritative not found

from:

could not determine result

Granularity and schema evolution

Caching an entire product object may seem convenient:

{
  "product_id": "chair-42",
  "name": "Gopher reading chair",
  "description": "...",
  "images": ["..."],
  "reviews": ["..."],
  "recommendations": ["..."]
}

Large cache entries consume:

• memory
• network bandwidth
• serialization time
• deserialization time

They may also combine fields with different lifecycles.

Updating one small price may require replacing a large object.

A more deliberate design might separate:

• product presentation
• price
• availability
• review summary

But splitting too aggressively creates more lookups and more coordination.

Cache granularity is another trade.

The useful unit is often the smallest coherent value with one ownership and freshness policy.

A cached object can outlive one application deployment.

Suppose version one stores:

{
  "price": 24900
}

Version two expects:

{
  "price_minor": 24900,
  "currency": "GBP"
}

During a rolling deployment, both application versions may read the same cache.

An old value can fail to decode in the new application.

A new value can confuse the old application.

Possible strategies include:

• versioned cache keys
• product:v1:chair-42
• product:v2:chair-42
• backward-compatible value changes
• short lifetimes during migration
• explicit cache flush

A cache does not eliminate schema evolution.

It creates another stored representation that may need versioning.

6. Failure and load behaviour

Eviction, cold starts, and stampedes

A cache has finite capacity.

When it needs space, it may remove entries according to an eviction policy.

Possible policies include variations of:

• least recently used
• least frequently used
• random selection
• soonest expiry

An entry can therefore disappear before its TTL ends.

entry is fresh
    │
    ▼
cache reaches memory limit
    │
    ▼
entry evicted

The next request becomes a miss.

This means a TTL is not a promise that the cache will retain the entry for that duration.

It is usually a limit on how long the entry remains eligible for reuse.

Capacity planning and eviction behaviour affect hit ratio and source load.

A cache should improve the source’s operating conditions.

It should not become the only thing preventing immediate collapse.

Caches are emptied by:

• restart
• deployment
• failure
• maintenance
• eviction
• configuration change
• mass invalidation

After a cold start:

CACHE

empty

Every request may reach the source.

many requests
      │
      ▼
many cache misses
      │
      ▼
database surge

The source and application need a cold-cache plan.

That may include:

• rate limiting
• request coalescing
• gradual warm-up
• preloading selected keys
• source capacity headroom
• stale fallback

A database that functions only while the cache is permanently warm is standing beneath a performance trapdoor.

Suppose a popular product cache entry expires.

At that moment, 500 requests arrive.

Without coordination:

500 requests
      │
      ▼
500 cache misses
      │
      ▼
500 identical database queries

This is often called a cache stampede or thundering herd.

The cache was intended to protect the database.

Its simultaneous miss now creates concentrated load.

The problem can be especially severe when:

• many keys share the same TTL
• a whole cache restarts
• a deployment invalidates a broad key range
• the source query is expensive

Expiration needs a concurrency policy.

Coalescing and TTL jitter

One mitigation is to allow one request to refresh a missing value while others wait for or reuse its result.

many callers
      │
      ▼
one loader selected
      │
      ├── read source
      └── repopulate cache

other callers
      │
      └── wait for shared result

This can be implemented with per-key locks, single-flight request groups, or shared promises and futures.

A process-local single-flight mechanism coordinates callers within one application instance. Ten application instances can still issue ten identical source requests. Distributed coalescing can coordinate loaders across instances, but it introduces shared-lock ownership, expiry and failure concerns.

The lock should usually be scoped to the cache key rather than the entire cache.

• product:chair-42 loading
• product:table-17 may load independently

The goal is to collapse duplicate work without serialising unrelated requests.

If thousands of entries are populated together with exactly the same TTL, they may expire together.

• 12:00 populate 10,000 entries
• 12:05 all 10,000 expire

Adding a small random variation can spread the reload work.

For example:

maximum TTL:
    300 seconds

random TTL:
    240 to 300 seconds

Entries then expire across a wider interval.

• entry A expires at 244 seconds
• entry B expires at 271 seconds
• entry C expires at 298 seconds

This does not solve every stampede.

It reduces synchronised expiry.

The freshness policy should account for the full possible TTL range.

Randomness should not silently extend sensitive values beyond their accepted stale window.

Hot keys and cache failure

A cache may remove pressure from the database while becoming the new bottleneck.

Suppose one key receives an enormous proportion of requests:

product:chair-42

All traffic reaches the same cache partition or server.

many application instances
          │
          ▼
       one hot key

Possible mitigations include:

• in-process caching above the shared cache;
• replication;
• request coalescing;
• partition-aware design;
• serving static content through a CDN.

An in-process cache can reduce pressure on the shared layer, but it also creates another freshness and invalidation boundary.

The correct solution depends on whether the pressure is:

• network traffic
• cache CPU
• connection count
• serialization cost
• single-partition throughput

A cache changes the bottleneck.

It does not guarantee the bottleneck disappears.

What should happen when the cache is unavailable?

Fail open to the source

cache unavailable
      │
      ▼
read database directly

Advantages:

• application remains functional
• cache stays an optimisation

Risks:

• database load surge
• latency increase
• outage moves downstream

Fail closed

cache unavailable
      │
      ▼
reject request

This may protect the source or preserve a security requirement.

It also turns cache availability into application availability.

Serve stale local data

shared cache unavailable
      │
      ▼
serve older process-local copy

This may suit selected read-only information.

It can be dangerous for sensitive or rapidly changing decisions.

The failure policy belongs to each use case.

“Cache unavailable” is not one universal condition with one universal response.

Timeouts protect the request path

A cache is intended to reduce latency.

If the application waits several seconds for an unhealthy cache before falling back to the database, the cache can make requests slower than having no cache.

The cache client needs:

• connection timeout
• operation timeout
• bounded retry policy

Retries deserve caution.

If the cache is overloaded, immediate retries multiply traffic against the failing component.

The application should not spend its entire request deadline trying to reach an optional optimisation.

A cache lookup may deserve a much smaller time budget than the complete request.

7. Ownership and consistency

Ownership, invalidation events, and versions

The application that owns the authoritative fact should normally control how changes are announced.

Suppose catalog owns product prices.

Catalog can:

• invalidate its cache directly
• publish ProductPriceChanged
• increment a product version
• update a materialised read model

Another service should not guess that a catalog value changed by examining catalog’s internal tables.

Ownership remains:

CATALOG

defines current product price
controls price changes
publishes supported change contract

A consumer may cache catalog data.

It must understand:

• how fresh that copy may be
• which change signal invalidates it
• what to do when invalidation fails

Caching does not transfer authority to the consumer.

Catalog might publish:

ProductPriceChanged

A consumer receives the event and removes:

product:chair-42

from its cache.

CATALOG
   │
   │ update price
   ▼
CATALOG DATABASE
   │
   │ publish event
   ▼
MESSAGE SYSTEM
   │
   ▼
CACHE INVALIDATOR
   │
   ▼
delete cached product

This can make invalidation responsive without every reader synchronously checking catalog.

But several failure modes remain:

• event was not published
• event delivery was delayed
• consumer was unavailable
• consumer processed events out of order
• wrong key was deleted
• new value was cached before old invalidation arrived

The event-driven design therefore needs reliable publication, duplicate handling, an ordering policy, replay, repair, and a TTL fallback.

The authoritative commit and event publication are still two operations unless they are coordinated. An outbox or database change stream can reduce the risk of committing a change without publishing its invalidation signal.

Events can improve invalidation.

They do not make invalidation atomic across independent systems.

Suppose every product change has a monotonically increasing version:

• version 17
• version 18
• version 19

The cached value includes:

{
  "product_id": "chair-42",
  "version": 19,
  "price_minor": 24900
}

If a delayed event for version 18 arrives, the cache updater can avoid replacing version 19 with older data.

incoming version 18
cached version 19
        │
        ▼
ignore older update

Versioning can help with out-of-order propagation.

It still requires the owner to define what the version orders:

• all product changes?
• price changes only?
• changes within one product?

A version is useful when its scope and comparison rules are explicit.

ACID and several versions of now

The authoritative database may provide an ACID transaction:

• BEGIN
• update product price
• record price history
• COMMIT

After commit, the database consistently contains the new price.

The cache remains outside that transaction.

DATABASE TRANSACTION
       │
       └── ACID boundary


CACHE UPDATE
       │
       └── separate operation

This means the database can be fully correct while the application serves a stale cache entry.

ACID protects the authoritative state.

Cache coherence requires another design.

The two should not be confused.

A cache is often where a strongly consistent write model meets an eventually refreshed read copy.

Imagine:

DATABASE
    price £249.00


SHARED CACHE
    price £239.00


APPLICATION INSTANCE A
    local cache price £229.00


BROWSER CACHE
    price £219.00

Each layer contains a value captured at a different moment.

• authoritative now
• shared-cache now
• process-local now
• browser-visible now

The system needs to decide where each version is acceptable.

This is especially important for write-after-read workflows.

A customer may see one price and submit an order later.

The checkout process should normally validate the current purchasable price rather than assuming that a cached product-page representation remains authoritative.

The cache can support browsing.

The owning service should make the final business decision.

Read-your-writes

Suppose a user updates their display name:

Miri → Mariam

The database commits the new value.

The application then redirects to a profile page whose data comes from a stale cache.

The user sees:

Miri

immediately after receiving confirmation that the update succeeded.

The system may be eventually consistent.

The experience feels broken.

Possible responses include:

• invalidate the user’s cache entry during the write
• update the cache with the committed value
• bypass cache for the immediate redirect
• return the new representation directly
• use a version or timestamp to reject older values

The required guarantee is sometimes called read-your-writes consistency:

After successfully changing a value, the same user should not immediately observe an older version.

Not every cached workflow needs this.

User-facing edits often do.

8. Operating and evaluating a cache

Measure the cache and its source

Useful cache measurements include:

• request count
• hit count
• miss count
• hit ratio
• latency
• error rate
• timeouts
• evictions
• memory usage
• connection usage
• key count
• expired entries

Application-level measurements may also include:

• source queries caused by misses
• load duration
• stale responses served
• invalidation failures
• stampede suppression
• age of cached values

Hit ratio needs context.

For example:

99% hit ratio

could be excellent.

It could also mean the cache stores one static key while every expensive query continues missing.

Metrics should be segmented by:

• cache purpose
• key group
• endpoint
• tenant
• result type

A single global percentage can conceal both hot successes and costly failures.

The cache may look healthy while the source quietly becomes incapable of handling misses.

A resilient design should understand:

• database latency on cache miss
• database traffic after cache restart
• cold-start behaviour
• maximum sustainable miss rate

Testing should include a cache-empty scenario.

warm cache test
    │
    └── normal steady state


cold cache test
    │
    └── restart, failover or mass invalidation

If the cold-cache test collapses the source, the architecture has an untested recovery cliff.

The cache’s recovery path is part of production design.

Warm-up and operational cost

Preloading every possible entry may be wasteful.

A cache may contain millions of products while most traffic targets a small popular subset.

Warm-up can focus on:

• highest-traffic products
• navigation data
• reference values
• known campaign pages
• recently active accounts

Another approach is gradual lazy loading combined with source protection.

The goal is not:

Make the cache completely full.

It is:

Prevent predictable miss traffic from overwhelming the source while useful entries repopulate.

A cache filled with values nobody requests is just expensive memory wearing running shoes.

A cache consumes more than infrastructure.

It adds code and operational reasoning around:

• keys
• serialization
• TTL
• eviction
• invalidation
• fallback
• concurrency
• monitoring
• failure recovery

A query that takes five milliseconds and runs 20 times per second may not need another distributed system in front of it.

Before adding a cache, I should establish:

• Which operation is slow?
• How often is it repeated?
• How much source load does it create?
• Can the source query be improved?
• Would an index solve the problem?
• Can the result safely become stale?
• What measurable improvement should the cache produce?

Sometimes the correct optimisation is:

• add an index
• remove an unnecessary query
• reduce response size
• batch related reads
• fix an N+1 query pattern

A cache can hide an inefficient source without fixing it.

That may be an acceptable tactical choice.

It should be recognised as one.

Cache the right work

Caching is most promising when data is:

• read frequently
• changed less frequently
• expensive to retrieve or compute
• safe to reuse temporarily

Examples might include:

• published product descriptions
• reference data
• rendered pages
• search suggestions
• feature metadata
• expensive aggregations

It is more difficult when data is:

• highly volatile
• security-sensitive
• unique to each request
• cheap to retrieve
• required to be immediately current

The decision should follow the ratio between:

• read frequency
• change frequency
• recomputation cost
• freshness requirement

Not every value deserves a cached afterlife.

9. A worked design and review contract

A simple product cache

Suppose catalog serves a product summary.

Requirements:

• product descriptions change infrequently
• price may change several times per day
• browse pages may tolerate up to 60 seconds of staleness
• checkout must validate current price directly
• catalog remains authoritative
• cache loss must not lose product data

A possible browse path is:

STOREFRONT
    │
    ▼
CATALOG API
    │
    ▼
PRODUCT CACHE
  │       │
 hit      miss
  │       │
  │       ▼
  │   CATALOG DATABASE
  │       │
  └───────┘

Possible policy:

key:
    catalog-product:v1:<product-id>:<currency>

TTL:
    60 seconds plus bounded jitter

population:
    cache-aside

write policy:
    commit database transaction,
    then invalidate affected keys

fallback:
    read database when cache fails,
    subject to source-protection controls

business validation:
    checkout reads authoritative current price

The design accepts:

browse price can lag briefly

while rejecting:

cached browse price is final checkout authority

The cache’s role is explicit.

Write down the cache contract

For each cached value, I should be able to document a clear contract:

If I cannot answer these questions, the cache is not yet a design.

It is a fast-looking box.

The mental model I am keeping

My original model was:

DATABASE
    │
    ▼
CACHE
    │
    ▼
faster application

The new model is:

                 AUTHORITATIVE SOURCE
                          │
                          │ copy
                          ▼
                        CACHE
                          │
        ┌─────────────────┼─────────────────┐
        │                 │                 │
        ▼                 ▼                 ▼
     faster reads     reduced source     possible
                          load            staleness
                                                │
                                                ▼
                                      freshness policy
                                                │
                         ┌──────────────────────┼───────────────┐
                         │                      │               │
                         ▼                      ▼               ▼
                        TTL               invalidation      validation

Around that copy sit several operational problems:

• misses
• eviction
• stampedes
• hot keys
• cold starts
• schema changes
• cache failure
• source overload

So caching is not simply:

Store the answer somewhere faster.

It is:

Create a managed copy whose speed, freshness, lifetime, identity and failure behaviour are appropriate for the operation using it.

That makes the trade more precise.

I am not always exchanging correctness for performance.

I am choosing a bounded freshness model for a particular read.

Sometimes the correct bound is:

five minutes

Sometimes it is:

five seconds

Sometimes it is:

validate before every reuse

And sometimes it is:

do not cache this decision

The fastest answer is not valuable merely because it arrived first.

It still has to be fresh enough to be true for the purpose in front of it.

References and further reading

HTTP caching

RFC 7234: HTTP/1.1 Caching Defines HTTP cache storage, freshness, validation, invalidation, private and shared caches, warning behaviour and reuse rules applicable when this article was written.

RFC 7232: HTTP/1.1 Conditional Requests Defines validators such as entity tags and modification dates, together with conditional headers including If-None-Match.

RFC 5861: HTTP Cache-Control Extensions for Stale Content Defines the stale-while-revalidate and stale-if-error cache-control extensions for serving stale responses during refresh or failure.

Redis caching behaviour

Redis Documentation: Key Expiration Documents assigning expiration times to Redis keys and the conditions under which expiry metadata is preserved or removed.

Redis Documentation: Eviction Policies Explains Redis memory limits and eviction policies, including least-recently-used, least-frequently-used, random and TTL-oriented selection.

Redis Documentation: Client-side Caching Describes keeping selected values in application memory and using invalidation messages to reduce the risk of stale local copies.

Memcached behaviour

Memcached Documentation Introduces Memcached as a distributed in-memory key-value cache and documents its server, client and operational model.

Memcached User Guide: Expiration Explains expiry values and the use of time-based removal for cached items.

Caching patterns

Microsoft Azure Architecture Center: Cache-Aside Pattern Describes application-managed cache lookup, source loading, population, expiry and invalidation, together with common suitability and consistency considerations.

Database query optimisation

PostgreSQL 13: Using EXPLAIN Explains how to inspect PostgreSQL query plans before treating caching as the only available performance solution.

PostgreSQL 13: Indexes Documents the role of indexes in accelerating database queries and the write and storage costs they introduce.