GPU passthrough on Proxmox, and why not an LXC

I wanted the GPUs in both Proxmox nodes for local models. An LXC container would have been lighter, but a container uses the host kernel, and the NVIDIA driver would not build against the Proxmox one. So both cards are passed through to VMs with VFIO. This post covers why, the host setup, and one backup it broke.

Why not an LXC container?

A GPU in a container is not virtualised. Containers on Proxmox use the host kernel directly, so the container uses the host’s NVIDIA driver. No driver on the host, no GPU in any container.

Proxmox VE 9.2 runs a 7.0 kernel. Debian 13, which it is built on, ships 6.12, and Debian’s packaged NVIDIA driver is the 550 series. Its DKMS module did not build against the Proxmox kernel. Lighter, shareable, fewer kernel flags, and it did not compile.

Passthrough moves the problem somewhere it does not exist. The host loads no graphics driver at all. The card is bound to vfio-pci and handed to a guest, and the guest runs an ordinary Ubuntu 24.04 kernel that the vendor driver supports.

Do you need amd_iommu=on?

No. The Proxmox docs say IOMMU is on by default with AMD CPUs, and with Intel CPUs on kernel 6.8 or newer. You still enable it in the BIOS.

I had amd_iommu=on on the kernel command line anyway, because a lot of guides have it. The kernel docs list no on value for amd_iommu. The parser logs Unknown option and moves on, so it sat there doing exactly nothing.

The flag worth keeping is iommu=pt, IOMMU passthrough mode, which the Proxmox docs say may improve performance. On a GRUB install it goes in /etc/default/grub:

GRUB_CMDLINE_LINUX_DEFAULT="quiet iommu=pt"

Then run update-grub. A node that boots with systemd-boot takes the command line from /etc/kernel/cmdline instead, applied with proxmox-boot-tool refresh.

Binding the card to vfio-pci

Load the VFIO modules at boot, in /etc/modules-load.d/vfio.conf. The earlier version of this post skipped this step:

vfio
vfio_iommu_type1
vfio_pci

Proxmox tries to take the card away from the host on its own. When that does not work, the fallback is to bind it by vendor and device ID in /etc/modprobe.d/vfio.conf:

options vfio-pci ids=10de:1f91,10de:10fa

Those are the GTX 1650 and its HDMI audio function, from lspci -nn. Both IDs, not one. The two functions share an IOMMU group, and VFIO will not use a group until every device in it is bound to VFIO.

Then keep the open source driver off the card, in /etc/modprobe.d/blacklist-nouveau.conf:

blacklist nouveau

Refresh the initramfs, reboot, and check both the IOMMU and the driver:

update-initramfs -u -k all
reboot
dmesg | grep -e DMAR -e IOMMU -e AMD-Vi
lspci -nnk -d 10de:        # want: Kernel driver in use: vfio-pci

If that line still says nouveau, nothing else in this post will work. If the blacklist is not enough, the Proxmox docs suggest a softdep so vfio-pci loads before the GPU driver.

Does your card reset cleanly?

Test this before you plan anything around the card. The GTX 1650 resets cleanly: stop the guest, start it again, and the card comes back without a host reboot. That guest can be restarted whenever I like.

Not every GPU does that. The Proxmox wiki lists cards with a reset bug, AMD Navi among them. The card does not reset properly, and the host has to reboot before the guest can start again.

What runs on the cards

The GTX 1650 has 4 GB of VRAM. Its guest is Ubuntu 24.04 on q35 and OVMF, with the vendor driver and the NVIDIA Container Toolkit so Docker can use the card. It runs Ollama and Open WebUI with a 3B model, which is about as big as 4 GB allows.

The RTX 5060 runs the bigger models, plus Whisper for speech to text and pyannote for speaker diarisation.

Three checks, inside the guest, not the host:

nvidia-smi
docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi
ollama ps                  # PROCESSOR column should say 100% GPU

The second one is the sample from NVIDIA’s toolkit docs. If it works, everything above it is correct. 100% CPU or a CPU/GPU split in the last one means the model did not fit in VRAM.

Check the backup on day one

Passthrough changes how backups behave. A stop mode vzdump shuts the guest down, then runs a background QEMU process to back up its data. That process starts from the guest’s config, GPU included.

On my primary node that process could not acquire the RTX 5060. The backup of that one guest failed while every other guest backed up fine. qm config will not tell you this. The backup job log will.

Snapshot mode backs up a running VM inside its existing QEMU process, so it should avoid the second start. I have not tested it on that guest yet.

What this does not cover

• Sharing one card between VMs (vGPU, mediated devices).
• NVIDIA’s own installer or a newer driver on the host. Debian’s packaged driver is the one that failed, and I did not try the others.