Case StudyHow Enkrypt AI ships its AI security platform into air-gapped KubernetesRead the case study
Skip to main content

Glossary

NVIDIA GPU Operator: Automated Kubernetes Management of GPU Software


What Is the NVIDIA GPU Operator?

The NVIDIA GPU Operator is a Kubernetes operator that installs and manages the software GPU nodes need. It deploys the driver, container toolkit, device plugin, node labeling, and monitoring as pods. You describe the desired state once, and the operator keeps every GPU node consistent. Pods then request GPUs without manual node setup.

It is open source under the Apache 2.0 licence and follows the standard Kubernetes operator pattern. A custom resource called ClusterPolicy holds the configuration.

Why does the GPU Operator matter for GPU clusters?

Because a GPU node needs several matching software layers before a container can use it. The driver, container runtime hook, and device plugin must all agree on versions. Doing this by hand across many nodes is slow and error-prone.

Version drift is the common failure. A kernel update breaks the driver build, or a new node joins without the toolkit. The operator removes that class of failure by treating GPU software as declarative, versioned state. It matters most on clusters that grow and upgrade often.

What components does the GPU Operator manage?

The operator runs one DaemonSet or controller for each layer of the stack.

ComponentRole
NVIDIA driver containerInstalls the kernel driver on the node
Container toolkitLets containerd or CRI-O expose GPUs to containers
Device pluginAdvertises nvidia.com/gpu resources to the kubelet
GPU Feature DiscoveryLabels nodes with GPU model, memory, and driver version
DCGM exporterExposes GPU metrics for Prometheus
MIG ManagerApplies MIG partitioning profiles from node labels
Node Feature DiscoveryDetects PCI hardware so GPU pods land on the right nodes

Optional components cover vGPU hosts, GPUDirect Storage, and confidential computing. Each can be toggled in the Helm values.

How does the GPU Operator work?

Node Feature Discovery labels nodes that contain NVIDIA hardware. The operator controller watches those labels and schedules its component pods only there. Components start in order, and a validator pod checks that the driver, toolkit, and plugin work before the node accepts GPU workloads.

Upgrades follow the same declarative path. You change the driver version in ClusterPolicy, and the operator rolls the change across nodes. The node-level approach is described in more detail for partitioning in optimizing NVIDIA GPUs with partitioning in Kubernetes.

How do you install the GPU Operator?

Installation is a Helm chart from NVIDIA. Add the repository, then install into a dedicated namespace:

helm repo add nvidia https://helm.ngc.nvidia.com/nvidia helm repo update helm install --wait gpu-operator -n gpu-operator --create-namespace nvidia/gpu-operator
bash

If the node image already includes a driver, set driver.enabled=false. Do the same for the toolkit when your distribution ships it. Confirm success by checking that the validator pods complete and that nodes report allocatable nvidia.com/gpu.

Can the GPU Operator share GPUs between workloads?

Yes, through two mechanisms. MIG Manager applies hardware partitions on supported cards, giving strong isolation. The device plugin can also time-slice a GPU, which shares one device with no memory isolation. For finer software-based sharing, see GPU fractioning. Virtualised GPUs with vendor licensing are covered in what is vGPU.

How CloudRaft helps

CloudRaft deploys and operates GPU stacks on Kubernetes, including operator configuration, driver upgrades, and GPU monitoring. Talk to our team.

Conclusion

The NVIDIA GPU Operator turns GPU node setup into declarative Kubernetes state. It installs and upgrades the driver, toolkit, device plugin, and monitoring consistently. Sharing options such as MIG and time-slicing build on top of it.