CUDA 13.4 can refuse a GPU allocation that crosses a memory limit

Watch

Primary

On October 27, 2026, NVIDIA said CUDA 13.4 enforces a per-workload GPU memory boundary. A job can ask for an exact size, and an allocation past that size fails. KAI Scheduler v0.18.0 can write that limit from Kubernetes. Time-slicing keeps each job in its own error domain. The parallel mode does not.

At a glance

Behavior is from NVIDIA's October 27 developer post. No cluster measurement is printed there.
CUDA	13.4
Memory request	An allocation above the cap fails
Burst	Optional maximum above a guaranteed minimum
Default compute	Time-slicing, separate error domain
Parallel compute	Sm-sharing, one shared error domain
Scheduler	KAI Scheduler v0.18.0, NvFractions mode
Operator	gpu-fractioning, open source, KAI Scheduler organization
Example	40 GiB guaranteed, 60 GiB burst, on each of four GPUs

How to read this claim

Measurement — The post does not publish a before-and-after utilization number.
Bill — List input and output per million tokens on the vendor page — a loop pays both again.
Door — Status on this card: Shipped.

Keep the habit · field card with prices and worked checks

What changed

Earlier sharing either isolated the GPU in hardware, took turns without a memory cap, or ran jobs together in MPS. CUDA 13.4 adds a memory cap that fails an oversized allocation, and a Kubernetes operator that applies it.

What it introduces

A GPU memory cap that CUDA enforces, with a Kubernetes operator that applies it per pod.

Who benefits Anyone packing more than one inference or training job onto a GPU, and anyone who assumed a shared GPU still kept a neighbor's crash contained.

Try this

If you share GPUs, set an exact memory request and treat the limit as the failure line. Use time-slicing when a crash in one job must not stop the others. Use sm-sharing only for jobs you are willing to put in the same error domain.

What we do not know

The post does not publish a before-and-after utilization number.
The post does not say which GPU generations support the new boundary.
Sm-sharing shares one error domain. Time-slicing does not.

Evidence

Primary Developer post. The memory behavior and the scheduler version are from NVIDIA's October 27 developer post.

Independent This is NVIDIA's description of CUDA 13.4 and KAI Scheduler v0.18.0. No outside cluster has published a measurement.

Sources

Developer post Primary · October 27, 2026

The record

What happened

NVIDIA's October 27 developer post says CUDA 13.4 adds a memory boundary that the application does not have to implement. A workload can request an exact memory size, and an allocation above it fails, as it would on a smaller GPU. It can also request a guaranteed minimum and a maximum it may grow into when memory is free. Each workload chooses time-slicing, the default, or sm-sharing. In time-slicing the jobs take turns and each has its own error domain. In sm-sharing they run at once and share one error domain, so a crash can affect the others. NVIDIA also published an open-source Kubernetes operator, gpu-fractioning, under the KAI Scheduler organization. KAI Scheduler v0.18.0 adds a GPU-sharing mode called NvFractions that writes the memory request, the limit, and the compute mode onto the pod. A GPU runs one compute mode at a time. NVIDIA's example is a 40 GiB guarantee on each of four GPUs, with room to grow to 60 GiB.

Why it matters

A cluster that gives every small job a whole GPU pays for idle memory. A cluster that shares a GPU without a cap pays when one job takes the rest. CUDA 13.4 is the cap. The part a platform team can get wrong is the compute mode: the faster shared mode does not isolate a crash.

If the burst limit is left off, NVIDIA says the limit equals the request and the job is capped at exactly what it asked for.

A portion request, such as half a GPU, can carry a portion limit, such as 0.8. NVIDIA says that portion limit has to be greater than the request and less than 1.

The operator measures memory and compute per fraction rather than per whole GPU. The post does not publish a utilization number from a cluster.