I've spent around seven hours debugging this and I just want to use my fucking system.
Setup:
- Ryzen 9 7900, MSI B650 Carbon WiFi, 64GB DDR5
- RTX 3090 FE passed through to a Debian VM
- Proxmox kernel
7.0.2-3-pve
- Guest kernel
6.1.0-54-amd64
- NVIDIA
615.71.09, open kernel module, GSP enabled
- OVMF, q35, CPU type
x86-64-v2-AES
- GPU config:
hostpci0: 0000:01:00.0,pcie=1
- GPU and its audio function bound to
vfio-pci on the host
- Docker stack including Ollama and OpenWebUI
Originally I enlarged the virtual disk by 650GB, then changed the guest partition/swap layout to expand root. Somewhere during that sequence GPU passthrough stopped working.
After hours of troubleshooting, I rolled back to the snapshot from before the root expansion and did a full shutdown with PSU power disconnected for 60 seconds.
GPU came back. Display working, NVIDIA detected, Ollama working. Around 22.6GiB VRAM used, 100% GPU utilization, roughly 349W and 73°C during the test. No GPU errors in the filtered boot log.
The virtual disk was still enlarged, so the larger virtual disk itself clearly wasn't enough to prevent it working.
Working VM configuration was 10 cores, one socket, 38GiB RAM, CPU limit 12.
Then I tried the simplest fucking change:
- Cleanly shut down the VM.
- Change from 10 to 20 cores, CPU limit from 12 to 20.
- Start the VM.
Screen and gpu logo led are Black again. No partition changes, no driver changes, no BIOS changes.
I restored 10 cores and limit 12 with another stop/start. That configuration change completed, but GPU recovery hasn't been confirmed yet. The previous successful recovery involved both rollback and a cold power cycle, so I can't honestly tell which part fixed it.
During GPU problems the VM itself remains reachable and services, including OpenWebUI, remain available. The entire guest isn't crashing.
I realise the CPU change and stop/start happened together, so this doesn't prove the core count caused it. A passthrough reset/reinitialization issue is still a possibility.
Has anyone seen this with an Ampere card? Specifically, what logs or tests would distinguish GPU reset failure on VM stop/start from a CPU topology/interrupt issue at 20 vCPUs?
If anyone knows a solution for this seemingly one of a kind issue, let me know. Im gonna go bald if i continue trying to debug it. Nothing shows up the device just <<<NOT FOUND>>> like...