Why CPU limits can make latency weird

The catalog service has this resource configuration:

resources:
  requests:
    cpu: "250m"
    memory: "256Mi"
  limits:
    cpu: "500m"
    memory: "512Mi"

Its average CPU usage is:

approximately 300m

The CPU limit is:

500m

Nothing appears alarming.

Then the latency dashboard develops occasional spikes:

p50:    18 ms
p95:    42 ms
p99:   190 ms
p99.9: 480 ms

The service is not crashing.

Memory is stable.

The database is behaving normally.

The node still has spare-looking CPU capacity.

Yet some requests periodically wait hundreds of milliseconds.

The CPU limit may be involved because Kubernetes is not giving the container half of one dedicated processor running continuously.

It is enforcing a CPU-time allowance through Linux control groups.

Kubernetes documents CPU limits as hard limits enforced by kernel throttling. When a container reaches its permitted CPU usage, the kernel restricts further execution rather than terminating the process.

CPU throttling does not necessarily make every request slightly slower. It can allow the service to run quickly, pause it completely and then let it continue.

That stop-start behaviour is where latency becomes strange.

`500m` is a throughput allowance

A CPU value of:

500m

means:

0.5 CPU

It does not necessarily mean the process runs on half of one physical core at every instant.

On Linux, a cgroup CPU limit is expressed as:

quota / period

With cgroup v2, this appears in cpu.max.

The kernel documentation describes it as the maximum amount of CPU time the group may consume during each accounting period. Kubernetes commonly uses a 100-millisecond period, although kubelet configuration can change it. The effective cgroup files on the running node remain the source of truth.

KUBERNETES YAML

declares the CPU limit

NODE AND CGROUP CONFIGURATION

determine the actual quota
and accounting period

Conceptually, with the common 100-millisecond period, a 500m limit can become:

CPU quota:   50 ms
Period:     100 ms

The container may consume up to 50 milliseconds of CPU time during each 100-millisecond wall-clock period.

That allowance can be spent in different shapes.

Smooth consumption

one thread runs
for 50 ms

then waits
for 50 ms

Parallel burst

two threads run
for 25 ms each

total CPU consumed:
50 ms

The second shape can exhaust the entire allowance after only 25 milliseconds of wall-clock time.

The cgroup may then be throttled for most of the remaining period.

0 ms                         100 ms
│                               │
├──── 25 ms busy ────┬──────────┤
                     │
                     └── throttled
                         until next period

A request arriving during the busy section may complete quickly.

A request becoming runnable immediately after the quota is exhausted may have to wait for the next period before the process can continue.

The two requests experience the same application code and dramatically different latency.

The node can look idle while the container waits

A CPU limit applies to the container’s cgroup.

It does not ask whether another processor on the node happens to be idle.

Suppose an eight-vCPU node is using only:

3 CPU

The catalog container has already exhausted its own 500m allowance for the current period.

The node may still have several idle processors.

The container cannot borrow them beyond its configured quota.

NODE

8 CPU available
3 CPU currently busy
5 CPU apparently free


CATALOG CONTAINER

period quota exhausted
therefore throttled

This can make dashboards confusing.

At the node level:

CPU capacity looks available.

At the container level:

The process is not permitted
to execute yet.

A CPU limit is an isolation boundary.

It deliberately prevents one workload from consuming unlimited spare capacity.

That can protect neighbouring workloads.

It can also leave usable node CPU stranded while a latency-sensitive request waits behind the container’s quota clock.

Average CPU hides the shape of consumption

A one-minute graph may show:

average CPU usage: 480m
CPU limit:          500m

That looks like the service remained just below its limit.

The average does not reveal whether consumption was:

steady at approximately 480m

or:

short bursts above one CPU
followed by enforced pauses

Both patterns may produce similar averages.

Their request latency can be very different.

PATTERN A

moderate CPU
moderate CPU
moderate CPU
moderate CPU


PATTERN B

large burst
throttled
large burst
throttled

The second pattern is common in work involving:

Protobuf encoding and decoding
TLS handshakes
JSON conversion at an edge
garbage collection
decompression
batch processing
message bursts
concurrent request fan-out
startup and readiness work

A Go service can be mostly I/O-bound and still produce short periods of parallel CPU demand.

The Go runtime itself also performs work such as garbage collection that can contribute to bursts. The Go team notes that CPU throttling can materially affect tail latency and that even applications with modest ordinary parallelism may encounter spikes from runtime work.

The important graph is therefore not merely:

CPU used

It is the relationship between:

CPU used

CPU throttling

request latency

garbage collection

request concurrency

Go can spend the allowance quickly

Go schedules many goroutines across a smaller collection of operating-system threads.

GOMAXPROCS controls how many threads may execute Go code simultaneously.

Before Go 1.25, the runtime normally chose GOMAXPROCS using the number of logical CPUs visible on the host.

A container limited to:

500m CPU

might run on a node with:

64 logical CPUs

The Go runtime could therefore attempt far more parallel execution than the cgroup allowance could sustain.

The process might consume its quota almost immediately and then be suspended.

Go 1.25 introduced a container-aware default on Linux. When the module uses Go 1.25-or-later language semantics and GOMAXPROCS has not been overridden, the runtime considers:

the visible logical-CPU count;
the process CPU-affinity mask;
the cgroup CPU throughput limit.

It periodically checks for changes. Kubernetes CPU requests are not used for this calculation.

The automatic behaviour is not guaranteed merely because the binary was built with a recent toolchain. It is disabled when:

the GOMAXPROCS environment variable is set;
the program calls runtime.GOMAXPROCS;
GODEBUG=containermaxprocs=0 disables cgroup-aware selection;
GODEBUG=updatemaxprocs=0 disables periodic updates.

For modules declaring Go 1.24 or earlier language semantics, the compatibility default also leaves container-aware selection disabled unless it is explicitly enabled.

BEFORE GO 1.25 DEFAULT

visible node CPUs
       │
       ▼
large GOMAXPROCS
       │
       ▼
rapid quota consumption
       │
       ▼
throttling


GO 1.25 CONTAINER-AWARE DEFAULT

visible CPUs
       │
       ├── CPU affinity
       └── cgroup CPU limit
                │
                ▼
       more suitable GOMAXPROCS

This reduces severe host-versus-container mismatches.

It does not make CPU limits and GOMAXPROCS equivalent.

It is also not guaranteed to improve every workload. Reducing runtime parallelism may trade less kernel throttling for slower completion of a short parallel burst. The relevant comparison is user-visible latency under the real workload.

Parallelism and throughput are different limits

A CPU limit controls total CPU time consumed during a period.

GOMAXPROCS controls simultaneous Go execution.

Suppose:

GOMAXPROCS = 2

CPU limit = 500m

Two Go threads may run at once.

If both remain busy, together they can consume a 50-millisecond quota in approximately 25 milliseconds of wall-clock time.

The kernel can then throttle the cgroup until the next period.

TWO RUNNING THREADS

25 ms × 2
=
50 ms CPU allowance consumed

This is particularly relevant for fractional CPU limits.

GOMAXPROCS must be an integer.

Go rounds a fractional cgroup CPU limit upwards when deriving parallelism. The runtime also normally avoids selecting a default below two unless the logical-CPU or affinity count is itself below two.

A 500m limit therefore does not normally lead to:

GOMAXPROCS = 1

With the Go 1.25 container-aware default, the runtime will normally select:

GOMAXPROCS = 2

when at least two logical or affinity-visible CPUs are available.

That is intentional. Go rounds the fractional limit upwards and normally maintains a minimum of two.

This may improve the service’s ability to perform concurrent work.

It can also make the short accounting-period behaviour visible.

Go 1.25 gives the runtime a better default.

It cannot turn half a CPU into a smooth half-speed processor.

Throttling can delay work that looks unrelated

When the cgroup is throttled, the kernel does not know which goroutine is important to the customer.

The paused work may include:

the handler serving the request
a goroutine returning a database connection
the garbage collector
a health check
a context-cancellation path
a metrics exporter
a Kafka heartbeat
graceful shutdown

Suppose a request has already received its database result.

It now needs only to:

construct the Protobuf response;
update a metric;
write bytes to the socket.

The container exhausts its CPU allowance between steps one and two.

The database trace looks healthy.

The network looks healthy.

The remaining application work waits for the next CPU period.

The user experiences a latency spike.

DATABASE RESPONSE
       │
       ▼
SMALL CPU BURST
       │
       ▼
QUOTA EXHAUSTED
       │
       ▼
CONTAINER THROTTLED
       │
       ▼
RESPONSE WRITE DELAYED

This is one reason distributed traces can show an unexplained gap between spans.

The application may not be blocked on a downstream service.

It may simply not be running.

More replicas do not always cure the shape

Horizontal scaling can reduce CPU demand per Pod.

It is often part of the answer.

But an autoscaler usually reacts after metrics have observed load.

A short burst may arrive before new replicas are ready.

The existing Pods can all exhaust their quotas together.

REQUEST BURST
      │
      ▼
ALL CURRENT PODS BUSY
      │
      ▼
ALL REACH CPU LIMIT
      │
      ▼
TAIL LATENCY RISES
      │
      ▼
HPA OBSERVES LOAD
      │
      ▼
NEW PODS START LATER

The CPU request also affects percentage-based HPA calculations.

A Pod using 400m CPU appears as:

160% utilisation

when request = 250m

but:

80% utilisation

when request = 500m

The process behaviour is identical.

The scaling interpretation changes.

This percentage model applies when the HPA uses a CPU-utilisation target. An HPA using a raw CPU value or a custom metric follows a different calculation.

The HPA is also an intermittent control loop rather than a continuous reaction. Its ordinary sync period is 15 seconds, and missing metrics, readiness handling and Pod startup time can delay or dampen scaling.

This makes it possible to combine:

an inaccurate request;
a tight CPU limit;
delayed autoscaling;
and confusing latency.

The YAML values are not independent knobs.

They form one operating model.

Removing CPU limits is not universally correct

It is tempting to conclude:

CPU limits cause throttling.

Therefore never set CPU limits.

That is too broad.

Without a CPU limit, a container can use CPU beyond its request when spare capacity exists.

This may improve latency for bursty workloads.

During node contention, CPU requests influence relative scheduling weight, so workloads that exceed their requests may receive less CPU when neighbours are busy.

A no-limit policy can work well when:

requests are measured honestly;
cluster utilisation is controlled;
namespace quotas constrain aggregate requests;
workloads are mutually trusted;
bursts are short;
latency is more important than rigid CPU ceilings;
excessive CPU use is monitored.

A CPU limit may still be appropriate when:

strong tenant isolation is required;
one runaway workload could harm many others;
predictable cost boundaries matter;
the service contains uncontrolled parallel work;
node sharing is aggressive;
the environment requires fixed resource ceilings.

The Go team describes the decision as a trade between using idle resources and obtaining more predictable container behaviour. It does not recommend one universal answer.

NO CPU LIMIT

better access to idle capacity
less hard throttling
larger noisy-neighbour risk


CPU LIMIT

stronger consumption boundary
more predictable maximum
possible throttling and tail latency

The choice should be tested under the actual node-sharing and traffic model.

Verify the runtime before explaining it

The intended Kubernetes configuration is not enough evidence.

Before attributing latency to CPU quota behaviour, I would inspect the running Pod and record:

effective GOMAXPROCS

cpu.max
or cgroup-v1 quota files

cpu.stat counters

CPU request and limit

Go module language version

GOMAXPROCS environment override

GODEBUG containermaxprocs
and updatemaxprocs settings

This answers an important preliminary question:

Is the process actually using
the container-aware default

or

has configuration disabled it?

A dashboard can describe the deployment manifest while the runtime is operating under a different parallelism or quota assumption.

What I would test for bfstore

I would run two related experiments.

The first should isolate the CPU quota.

The second should compare realistic production profiles.

Experiment A: isolate the CPU limit

Keep the CPU request and GOMAXPROCS fixed while changing only the limit.

Conceptually:

profiles:
  - name: no-cpu-limit
    cpu_request: 250m
    cpu_limit: none
    gomaxprocs: 2

  - name: 500m-limit
    cpu_request: 250m
    cpu_limit: 500m
    gomaxprocs: 2

  - name: one-cpu-limit
    cpu_request: 250m
    cpu_limit: "1"
    gomaxprocs: 2

This experiment asks:

What changes when only
the cgroup quota changes?

The Pods should otherwise use:

the same application build;
the same memory configuration;
the same node class;
the same replica count;
the same workload;
the same HPA state, preferably disabled for the causal test.

Fixing GOMAXPROCS removes one major confounding variable.

Keeping the CPU request constant avoids changing scheduling weight and percentage-based HPA interpretation at the same time.

Experiment B: compare complete operating profiles

The deployment decision still needs realistic end-to-end configurations.

I would compare:

250m request
no CPU limit
automatic GOMAXPROCS


250m request
500m CPU limit
automatic GOMAXPROCS


250m request
1 CPU limit
automatic GOMAXPROCS


250m request
no CPU limit
explicitly selected GOMAXPROCS

This experiment asks:

Which complete operating profile
gives the best real outcome?

I would also test a profile with equal CPU request and limit when production capacity planning requires it.

Equalising CPU request and limit does not by itself create a Kubernetes Guaranteed Pod. Every container must also have equal memory request and limit, as well as equal CPU request and limit.

Across both experiments, I would compare:

throughput;
p50 latency;
p95 latency;
p99 latency;
throttled-period ratio;
accumulated throttled time;
CPU pressure;
garbage-collection CPU;
goroutine count;
HPA behaviour where enabled;
node utilisation;
neighbouring workload latency.

The test should include bursts.

A constant request rate may hide the very behaviour the CPU limit introduces.

STEADY TEST

reveals sustained capacity

BURST TEST

reveals quota exhaustion
and tail-latency shape

For bfstore, useful burst scenarios include:

a product launch;
simultaneous basket updates;
Kafka redelivery after an outage;
a deployment warming new Pods;
a payment-provider retry wave;
large Protobuf messages;
trace export recovery after an observability interruption.

The correct CPU policy is the one that protects the platform while preserving the service’s latency objective under those conditions.

The signals I would watch

At the application level:

request duration

request concurrency

requests per second

gRPC status codes

deadline exceeded errors

queue depth

Kafka consumer lag

At the Go runtime level:

effective GOMAXPROCS

garbage-collection CPU

GC pause and cycle frequency

goroutines

heap allocation rate

At the container level:

CPU usage rate

throttled-period ratio

increase in throttled time

CPU pressure

configured quota and period

CPU request and limit

With cgroup v2, cpu.stat includes counters such as:

nr_periods

nr_throttled

throttled_usec

A useful first normalisation is:

throttled-period ratio

=

increase in nr_throttled
/
increase in nr_periods

Raw throttled time is not the same as request delay.

It is scheduler evidence, not an application-latency measurement.

The useful question is not merely:

Did throttling occur?

A small amount may be harmless.

The stronger question is:

Did throttling increase
for the serving Pod

at the same time as
that Pod's request latency?

A service-wide p99 can rise because one replica is throttled while aggregate CPU remains smooth across all replicas.

REQUEST
   │
   ▼
SERVING POD
   │
   ├── request latency
   ├── CPU usage
   ├── throttled-period ratio
   ├── GOMAXPROCS
   └── runtime activity

Aggregate dashboards are useful for detection.

Per-Pod evidence in the same short time window is stronger for attribution.

Correlation is not automatically causation.

It is an excellent invitation to rerun the workload with a controlled change to the CPU quota or runtime profile.

The mental model I am keeping

My earlier model was:

CPU LIMIT = 500m

the service receives
half of a CPU

The stronger model is:

                     CPU LIMIT
                         │
                         ▼
               TIME-BASED CGROUP QUOTA
                         │
             ┌───────────┴───────────┐
             │                       │
             ▼                       ▼
       CPU CONSUMED SLOWLY      CPU CONSUMED IN BURST
             │                       │
             ▼                       ▼
       SMOOTHER PROGRESS       QUOTA EXHAUSTED EARLY
                                     │
                                     ▼
                              CGROUP THROTTLED
                                     │
                                     ▼
                              GOROUTINES PAUSE
                                     │
                                     ▼
                              TAIL LATENCY RISES

The request influences placement and contention weight.

The limit defines periodic CPU-time throughput.

GOMAXPROCS limits simultaneous Go execution.

Throttling shows that the quota was exhausted.

Latency shows whether users cared.

A CPU limit can make latency weird because it is not a continuous speed dial.

It is a recurring allowance.

The service may sprint, spend its allowance and be ordered to stand still while the request timer continues walking.

Average CPU can remain reasonable.

The node can retain idle capacity.

The process can remain healthy.

The latency tail can still grow teeth.

Go 1.25 can reduce one important source of excessive parallelism when the container-aware default is active.

It may also reduce useful short-lived parallelism for some bursty workloads.

Fractional limits, bursty work and short cgroup periods still require measurement.

For bfstore, I would not adopt either always set CPU limits or never set CPU limits as a golden-path commandment.

I would provide:

an honest CPU request;
a measured default profile;
Go 1.25’s container-aware runtime behaviour;
throttling and latency dashboards;
burst-load tests;
an evidence-based choice about the limit.

The CPU limit should protect the platform.

It should not protect the service so vigorously that the service cannot finish speaking.

References and further reading

Kubernetes: Resource management for Pods and containers Defines CPU requests and limits and explains kernel-enforced CPU throttling.
Kubernetes: Kubelet configuration Documents the default and configurable CPU CFS quota period.
Kubernetes: Horizontal Pod Autoscaling Explains utilisation targets, request-relative calculations, the control loop and readiness handling.
Kubernetes: Pod quality-of-service classes Defines the CPU and memory requirements for the Guaranteed QoS class.
Kubernetes: CPU management policies Explains the default CPU Manager policy and the optional static policy for workloads requiring stronger CPU affinity.
Go: Container-aware GOMAXPROCS Explains cgroup CPU quotas, accounting periods, throttling, tail latency and the trade-offs in Go’s container-aware runtime behaviour.
Go 1.25 release notes Documents the container-aware default, automatic updates and the settings that disable them.
Go: GODEBUG defaults and compatibility Explains how module language versions and GODEBUG settings control container-aware GOMAXPROCS.
Go runtime documentation Documents how the runtime derives GOMAXPROCS from logical CPUs, affinity and cgroup throughput limits, including fractional rounding.
Linux cgroup v2 documentation Defines cpu.max, cpu.stat, quota periods, throttling counters and CPU pressure.

500m is a throughput allowance