Single-GPU host: VFIO claim fails with "invalid spec, no devices"

[poussa]

Description

On a host with exactly one GPU, preparing a gpu-vfio.intel.com claim fails and the device is left stranded on vfio-pci. Multi-GPU hosts are unaffected.

FailedPrepareDynamicResources ... failed to prepare VFIO device 0000:03:00.0:
  failed to add device to CDI registry: failed to remove old GPU devices from
  CDI spec: failed to write CDI spec intel.com-gpu.yaml:
  invalid CDI Spec: invalid spec, no devices
Error: ... CDI device injection failed: failed to stat CDI host device
  "/dev/dri/card0": no such file or directory

Cause

Binding the GPU to vfio-pci removes its DRM nodes, so prepareVFIODevice → UpdateGPUDevices → RemoveDevices drops that device from the CDI spec. With one GPU that empties devices, and CDI rejects a spec with no devices.

RemoveDevices (pkg/gpu/cdihelpers/cdihelpers.go) calls cdiCache.WriteSpec directly rather than the package's writeSpec helper, which already guards this:

func writeSpec(cdiCache *cdiapi.Cache, spec *specs.Spec) error {
	if len(spec.Devices) == 0 {
		return nil
	}

replaceGPUCDISpecs handles the same situation by removing the spec (covered by the existing "Existing specs, all devices removed" test), so the guard looks like an oversight in RemoveDevices rather than intent.

The write fails, so the stale entry is never removed and containerd then fails to stat the now-absent /dev/dri/card0. NodeUnprepareResources never completes, leaving the device on vfio-pci with no DRM nodes. Recovery:

echo "" | sudo tee /sys/bus/pci/devices/<addr>/driver_override
sudo sh -c "echo <addr> > /sys/bus/pci/drivers/vfio-pci/unbind"
sudo sh -c "echo <addr> > /sys/bus/pci/drivers_probe"

The reverse direction has the same defect: once the spec holds only the VFIO entry, a gpu.intel.com claim empties it again.

Reproduce

Any single-GPU host: claim one device from gpu-vfio.intel.com. As a unit test, the existing TestUpdateGPUDevices case with one device instead of two fails on the error above.

Environment

• Driver: chart 0.11.0 (v0.11.0); also present on main at 4e13373
• Kubernetes v1.36.3, KubeVirt v1.9.0
• Arc Pro B50 (8086:e212) and B60 (8086:e211)
• Confirmed failing on two single-GPU hosts, working on a 2-GPU host