Ubuntu 22.04 GPU passthrough (QEMU)

Question

I have found a guide that tells me to do the following to get GPU pass through working on 20.04

apt install qemu-kvm libvirt-clients libvirt-daemon-system bridge-utils virt-manager ovmf

lspci -nn returns

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA104 [GeForce RTX 3060 Ti Lite Hash Rate] [10de:2489] (rev a1)
01:00.1 Audio device [0403]: NVIDIA Corporation GA104 High Definition Audio Controller [10de:228b] (rev a1)

/etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash intel_iommu=on kvm.ignore_msrs=1 vfio-pci.ids=10de:2489,10de:228b"

Then

update-grub

However when I reboot back I get a plain black screen. Is the process different for 22.04? i9-12900k is the CPU

TACHMALEX · Answer 1 · 2024-09-13T10:55:05.573

GPU PASSTHROUGH GUIDE FOR UBUNTU WITH NOUVEAU DRIVER INSTALLED

This guide was tested on Ubuntu 22.04 LTS, 22.10, 23.04, 23.10, 24.04 LTS with Nouveau driver and the following system configuration:

MSI® motherboard Z170A
Intel® Core™ i5-6600 Processor
GeForce GTX 1650 AERO ITX 4G OC.

Attention! This guide is only relevant for Nouveau driver. Please read the text carefully before you start system changes. For ubuntu with the proprietary nvidia driver installed, see GUIDE: GPU PASSTHROUGH GUIDE FOR UBUNTU WITH NVIDIA PROPRIETARY DRIVER INSTALLED.

Disclaimer: This guide was written for research purposes. The author have no responsibility for any consequences, including potential device brick, violating manufacturer licenses, lose device warranty etc.

HARDWARE SPECIFICATION REQUIRED FOR THIS GUIDE

For Intel based system:

Motherboard with Integrated Graphics (for manipulations)
CPU with VT-x, VT-d and Processor Graphics
Discrete Graphics Card with GPU passthrough support (for passthrough)
Monitor with two inputs like Displayport, D-SUB, DVI, HDMI or switcher box with monitor or two monitors.

For AMD based system:

Motherboard with Integrated Graphics (for manipulations)
CPU with AMD-V, AMD-Vi and Integrated Graphics
Discrete Graphics Card with GPU passthrough support (for passthrough)
Monitor with two inputs like Displayport, D-SUB, DVI, HDMI or switcher box with monitor or two monitors.

See list of IOMMU-supporting hardware (list may not be complete).

Important: To improve guest machine productivity, install the guest operating system on an additional physical disk (passthrough solution). Also You will need 8GB RAM for operating system (OS) and 8GB RAM for guest machine, the 16GB RAM may significantly increase your HDD's or SSD's performance and lifetime.

REQUIRED STEPS

STEP 1. Enable Hardware-assisted virtualization in the BIOS

For Intel based system enable Intel Virtualization Tech (VT-x) and Intel VT-D Tech
For AMD based system enable SVM Mode.

STEP 2. Set the BIOS to use the Integrated Graphics as the primary boot device

Attention! If you boot the system with designed for passthrough PCI-Express Graphics Device, error code 43 will appear.

STEP 3. Check for IOMMU Support on your CPU

For AMD processor:

$ cat /proc/cpuinfo | grep --color svm

For Intel processor:

$ cat /proc/cpuinfo | grep --color vmx

You should see the highlighted text svm or vmx.

STEP 4. Enable IOMMU

$ sudo nano /etc/default/grub

Add the following options to GRUB_CMDLINE_LINUX_DEFAULT=""

For AMD processor:

amd_iommu=on kvm.ignore_msrs=1

For Intel processor:

intel_iommu=on kvm.ignore_msrs=1

STEP 5. Update GRUB

$ sudo grub-mkconfig -o /boot/grub/grub.cfg

STEP 6. Reboot your system

STEP 7. Check that IOMMU is enabled

$ sudo dmesg | grep -i -e DMAR -e IOMMU

You should see message like below:

[    0.123456] pci 0000:09:00.0: Adding to iommu group 1
[    0.234567] pci 0000:0a:00.0: Adding to iommu group 2
[    0.345678] DMAR: Intel(R) Virtualization Technology for Directed I/O

STEP 8. Find your device

$ lspci -nnk

You should see text like below:

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU117 [GeForce GTX 1650] [10aa:10bb] (rev a1) (prog-if 00 [VGA controller])
    Subsystem: Micro-Star International Co., Ltd. [MSI] TU117 [GeForce GTX 1650] [0101:a1a1]
    Flags: bus master, fast devsel, latency 0, IRQ 151, IOMMU group 1
    Memory at de000000 (32-bit, non-prefetchable) [size=16M]
    Memory at c0000000 (64-bit, prefetchable) [size=256M]
    Memory at d0000000 (64-bit, prefetchable) [size=32M]
    I/O ports at e000 [size=128]
    Expansion ROM at 000c0000 [disabled] [size=128K]
    Capabilities: <access denied>
    Kernel driver in use: nouveau
    Kernel modules: nvidiafb, nouveau
01:00.1 Audio device [0403]: NVIDIA Corporation Device [01cc:01ee] (rev a1)
    Subsystem: Micro-Star International Co., Ltd. [MSI] Device [0202:a2a2]
    Flags: bus master, fast devsel, latency 0, IRQ 17, IOMMU group 1
    Memory at df080000 (32-bit, non-prefetchable) [size=16K]
    Capabilities: <access denied>
    Kernel driver in use: snd_hda_intel
    Kernel modules: snd_hda_intel

If You see text "Kernel driver in use: nvidia" like below:

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU117 [GeForce GTX 1650] [10aa:10bb] (rev a1)
    Subsystem: Micro-Star International Co., Ltd. [MSI] TU117 [GeForce GTX 1650] [0101:a1a1]
    Kernel driver in use: nvidia
    Kernel modules: nvidiafb, nouveau, nvidia_drm, nvidia
01:00.1 Audio device [0403]: NVIDIA Corporation Device [01cc:01ee] (rev a1)
    Subsystem: Micro-Star International Co., Ltd. [MSI] Device [0202:a2a2]
    Kernel driver in use: snd_hda_intel
    Kernel modules: snd_hda_intel

your system uses NVIDIA Proprietary Drivers. You need to remove all old video drivers. Here is a recipe which removes all old video drivers, and reinstalls nouveau:

$ sudo nvidia-settings --uninstall
$ sudo apt-get remove --purge nvidia*
$ sudo apt-get remove --purge xserver-xorg-video-nouveau
$ sudo apt-get remove --purge xserver-xorg-video-nv
$ sudo apt-get install nvidia-common
$ sudo apt-get install xserver-xorg-video-nouveau
$ sudo apt-get install xserver-xorg-video-all
$ sudo apt-get install --reinstall libgl1-mesa-glx libgl1-mesa-dri
$ sudo apt-get install --reinstall xserver-xorg-core
$ sudo dpkg-reconfigure xserver-xorg

or run the following command:

$ software-properties-gtk --open-tab=4

select the option “X.Org X server -- Nouveau display driver from xserver-xorg-video-nouveau (open source)”

Run autoremove:

$ sudo apt autoremove

Then reboot your system and repeat the STEP 8.

STEP 9. Create a new file called vfio.conf

$ sudo nano /etc/modprobe.d/vfio.conf

Add the following lines with your device IDs from the STEP 8:

blacklist nouveau
blacklist snd_hda_intel
options vfio-pci ids=10aa:10bb,01cc:01ee

STEP 10. Update the existing initramfs

$ sudo update-initramfs -u

STEP 11. Reboot your system

STEP 12. Make sure everything is OK

$ lspci -nnk

You should see text like below, the lines “Kernel driver in use: nouveau” and “Kernel driver in use: snd_hda_intel” should not be present in the text:

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU117 [GeForce GTX 1650] [10aa:10bb] (rev a1)
    Subsystem: Micro-Star International Co., Ltd. [MSI] TU117 [GeForce GTX 1650] [0101:a1a1]
    Kernel modules: nvidiafb, nouveau
01:00.1 Audio device [0403]: NVIDIA Corporation Device [01cc:01ee] (rev a1)
    Subsystem: Micro-Star International Co., Ltd. [MSI] Device [0202:a2a2]
    Kernel modules: snd_hda_intel

STEP 13. Use VirtManager to create a new guest machine

Install KVM

$ sudo apt-get install qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils

Add user

Note: For newer versions of Linux kernels, try libvirt.

$ sudo adduser username libvirtd

If you have group issue "adduser: The group `libvirtd' does not exist." try:

$ sudo addgroup libvirtd

Install VirtManager

$ sudo apt-get install virt-manager

Create new guest machine with a custom configuration.

$ virt-manager

If you get a "make sure the libvirtd daemon is running` error, reboot and try again.

Optional: To install the guest operating system on an additional physical disk, uncheck the box "Enable storage for this virtual machine"

You must:

select the "Q35" chipset and "UEFI
x86_64:/usr/share/OVMF/OVMF_CODE_4M.ms.fd" firmware or similar

add PCI Host Devices for GPU passthrough

Optional: To install the guest operating system on an additional physical disk, select Host Device – physical SSD M.2 or SATA 3.0 PCIe Gen3 x1 card for SATA SSD/HDD

set QXL video driver and SPICE client for display with "Address"
listen type.

STEP 14. Edit the guest machine

$ virsh list --all
$ sudo virsh edit YourGuestMachineName

Add the following lines:

<vendor_id state='on' value='1234567890ab'/>
<kvm>
 <hidden state='on'/>
</kvm>
<ioapic driver='kvm'/>

The end result should look something like:

<features>
  <acpi/>
  <apic/>
  <hyperv>
    <relaxed state='on'/>
    <vapic state='on'/>
    <spinlocks state='on' retries='8191'/>
    <vendor_id state='on' value='1234567890ab'/>
  </hyperv>
  <kvm>
    <hidden state='on'/>
  </kvm>
  <vmport state='off'/>
  <ioapic driver='kvm'/>
</features>

Once your done editing, use CTRL+x to save the changes.

STEP 15. Run the guest machine

Optional: If you are installing the guest operating system on an additional physical disk, start the guest machine and hit the BIOS entry key F2 at the appropriate time. Set the physical disk device as bootable.

NOTE

If you want to use the PCI-Express Graphics Device as the primary boot device, do the following:

$ sudo nano /etc/modprobe.d/vfio.conf

Change the following lines with your device IDs from the STEP 8:

blacklist nouveau
blacklist snd_hda_intel
options vfio-pci ids=10aa:10bb,01cc:01ee

with:

#blacklist nouveau
#blacklist snd_hda_intel
#options vfio-pci ids=10aa:10bb,01cc:01ee

Update the existing initramfs

$ sudo update-initramfs -u

Reboot your system and set the BIOS to use the PCI-Express Graphics Device as the primary boot device.

TROUBLESHOOTING

See GPU PASSTHROUGH TROUBLESHOOTING

REFERENCES

TACHMALEX · Answer 2 · 2024-10-05T10:00:14.053

GPU PASSTHROUGH GUIDE FOR UBUNTU WITH NVIDIA PROPRIETARY DRIVER INSTALLED

This guide was tested on Ubuntu 23.10, 24.04 LTS, 24.04.1 LTS with NVIDIA Proprietary Drivers and the following system configuration:

MSI® motherboard Z170A
Intel® Core™ i5-6600 Processor
GeForce GTX 1650 AERO ITX 4G OC.

Attention! This guide is only relevant for Intel CPU based systems. Please read the text carefully before you start system changes.

Disclaimer: This guide was written for research purposes. The author have no responsibility for any consequences, including potential device brick, violating manufacturer licenses, lose device warranty etc.

HARDWARE SPECIFICATION REQUIRED FOR THIS GUIDE

Motherboard with Integrated Graphics (for manipulations)
Intel CPU with VT-x, VT-d and Processor Graphics
Discrete NVIDIA Graphics Card with GPU passthrough support (for passthrough)
Monitor with two inputs like Displayport, D-SUB, DVI, HDMI or switcher box with monitor or two monitors.

REQUIRED STEPS

STEP 1. Enable Hardware-assisted virtualization in the BIOS

Enable Intel Virtualization Tech (VT-x) and Intel VT-D Tech

STEP 2. Set the BIOS to use the Integrated Graphics as the primary boot device (important)

STEP 3. Check for IOMMU Support on your Intel CPU

$ cat /proc/cpuinfo | grep --color vmx

You should see the highlighted text vmx.

STEP 4. Enable IOMMU

$ sudo nano /etc/default/grub

Add the following options to GRUB_CMDLINE_LINUX_DEFAULT="":

intel_iommu=on kvm.ignore_msrs=1

STEP 5. Update GRUB

$ sudo grub-mkconfig -o /boot/grub/grub.cfg

STEP 6. Reboot your system

STEP 7. Check that IOMMU is enabled

$ sudo dmesg | grep -i -e DMAR -e IOMMU

You should see message like below:

[    0.123456] pci 0000:09:00.0: Adding to iommu group 1
[    0.234567] pci 0000:0a:00.0: Adding to iommu group 2
[    0.345678] DMAR: Intel(R) Virtualization Technology for Directed I/O

STEP 8. Install NVIDIA Proprietary Drivers

$ software-properties-gtk --open-tab=4

select the option “Using NVIDIA driver metapackage from nvidia-driver-XXX (proprietary, tested)”

STEP 9. Install nvidia-prime

$ sudo apt install nvidia-prime

STEP 10. Reboot your system

STEP 11. Set the Integrated Graphics as the primary

$ sudo prime-select intel

STEP 12. Check if the integrated graphics is set as primary

$ sudo prime-select query

You must see “intel”

STEP 13. Reboot your system

STEP 14. Use VirtManager to create a new guest machine

See STEP 13 from the GUIDE: GPU PASSTHROUGH GUIDE FOR UBUNTU WITH NOUVEAU DRIVER INSTALLED

STEP 15. Edit the guest machine

See STEP 14 from the GUIDE: GPU PASSTHROUGH GUIDE FOR UBUNTU WITH NOUVEAU DRIVER INSTALLED

STEP 16. Run the guest machine

NOTE

If you want to use the NVIDIA Graphics Device as the primary boot device, do the following:

Set the NVIDIA Graphics Device as the primary

$ sudo prime-select nvidia

Check if the NVIDIA Graphics Device is set as primary

$ sudo prime-select query

You must see “nvidia”

Reboot your system

$ sudo reboot

TROUBLESHOOTING

See GPU PASSTHROUGH TROUBLESHOOTING

REFERENCES

TACHMALEX · Answer 3 · 2024-09-13T10:55:14.070

GPU PASSTHROUGH TROUBLESHOOTING

The troubleshooting guide have been tested on different releases of Ubuntu. Please create a bootable Ubuntu Live USB/CD/DVD before you start system changes.

Disclaimer: This troubleshooting guide was written for research purposes. The author have no responsibility for any consequences, including potential device brick, violating manufacturer licenses, lose device warranty etc.

Problem: I installed the Nouveau driver. Instead of the login screen, I see a black screen with a mouse cursor on my primary or secondary monitor.

Solution 1. Action:

Set the BIOS to use the Discrete Graphics Card as the primary boot device
Boot the Ubuntu operating system
Edit the configuration file

$ sudo nano '/etc/gdm3/custom.conf'

Uncomment the line “# WaylandEnable=false” to force the login screen to use Xorg

Use CTRL+x to save the changes

Reboot your system
Set the BIOS to use the Integrated Graphics as the primary boot device
Boot the Ubuntu operating system.

Note: To undo the change, comment (#) the line “WaylandEnable=false”.

Solution 2. Action:

Set the BIOS to use the Discrete Graphics Card as the primary boot device
Boot the Ubuntu operating system
Create a new file named bl_nvidia.conf

$ sudo nano '/usr/lib/modprobe.d/bl_nvidia.conf'

Add following lines:

blacklist nvidia
blacklist nvidia-drm
blacklist nvidia-modeset
alias nvidia off
alias nvidia-drm off
alias nvidia-modeset off
blacklist nouveau
blacklist lbm-nouveau
alias nouveau off
alias lbm-nouveau off

Use CTRL+x to save the changes

Reboot your system
Set the BIOS to use the Integrated Graphics as the primary boot device
Boot the Ubuntu operating system.

Note: When using solution 2, Proprietary and Nouveau drivers for NVIDIA Graphics Devices will not load.

Note: To undo the change, delete the bl_nvidia.conf file.

$ sudo rm '/usr/lib/modprobe.d/bl_nvidia.conf'

Problem: Gnome login screen is not displayed after rebooting the system.

Action:

It is possible that your system is booting with integrated graphics, while the vaio.conf file is missing, is empty, has incorrect text or the lines described in step 9 start with # character. Check step 9.

Problem: The emulator may not have search permissions for the path '/*.qcow2'

Action:

Run virt-manager with sudo privileges or use chown, chgrp, chmod commands to fix. See the STEPS from this GUIDE.

Problem: Error starting domain: Cannot access storage file '/*.qcow2' (as uid:10101, gid:101): Permission denied

Action:

Run virt-manager with sudo privileges or use chown, chgrp, chmod commands to fix. See the STEPS from this GUIDE.

Problem: The guest machine start normally but I see a black screen on the output of PCI-Express Graphics Device.

Action:

check the cable connections and the settings of your monitor
check the steps 1, 2, 14
install the correct device driver on Windows
uninstall and reinstall the device driver on Windows
update the device driver on Windows

Problem: When using a USB redirector, devices such as a webcam, USB audio interface, USB HDD/SSD not work correctly.

Action:

Use an additional PCIe USB card, necessarily with IOMMU-support (PCIe card passthrough solution).

Problem: Intermittent sound with crackling on guest machine with Windows 10 operating system.

Action:

use generic kernel
Make sure that the Tjunction max value has not been exceeded
Disable WiFi in Ubuntu
to improve guest machine performance, set CPU governor on Ubuntu operating system to "Performance"
if you are using the HDA (ICH9) or HDA (ICH6) model with built in sound card set the number of vCPUs to half the number of CPUs of the logical host
if you are using an additional PCIe Gen3 x4 USB 3.2 card (PCIe card passthrough solution) with an USB audio interface, set processor topology manually (sockets, cores, threads) referring to physical CPU topology
if you are using an additional PCIe Gen3 x4 USB 3.2 card (PCIe card passthrough solution) with an USB audio interface, try different audio interface driver version
if the operating system Ubuntu has been converted to Ubuntu Studio, try reverting all changes
install clean Ubuntu operating system and try again.

Problem: The CPU governor "Performance" profile was available, but has now disappeared.

Action (Experimentally):

Find available Generic Kernel Images and install one of them

$ sudo apt list linux-*image-* | grep generic
$ sudo apt-get install linux-image-x.x.x-01-generic

Note: The x.x.x-01-generic and x.x.x-05-generic texts are used for examples.

Reboot your system
Keep hitting Shift until you see "Grub Loading Message". Through the "Advanced options for Ubuntu" menu, select and boot the installed kernel (linux-image-x.x.x-01-generic)
Use VirtManager to create a new guest machine and do a fresh installation of Ubuntu. Note that ideally the Ubuntu release and updates must be identical to those installed on the host machine
On guest machine run nautilus as root

$ sudo nautilus

Compress the files

/boot/config-x.x.x-05-generic
      initrd.img-x.x.x-05-generic
      System.map-x.x.x-05-generic
      vmlinuz-x.x.x-05-generic

and directory

/lib/modules/x.x.x-05-generic

in .tar.xz format

Copy the .tar.xz files to the host machine
Run nautilus as root on the host machine. Delete files

/boot/config-x.x.x-05-generic
      initrd.img-x.x.x-05-generic
      System.map-x.x.x-05-generic
      vmlinuz-x.x.x-05-generic

and directory

/lib/modules/x.x.x-05-generic

from the physical hard drive. Then extract the .tar.xz files to the appropriate directories

Reinstall the kernel image

$ sudo apt-get -f reinstall linux-image-x.x.x-05-generic

List installed kernel

$ find /boot/vmli*

You should get similar text:

/boot/vmlinuz
/boot/vmlinuz-x.x.x-01-generic
/boot/vmlinuz-x.x.x-05-generic
/boot/vmlinuz.old

Reboot your system
Keep hitting Shift until you see "Grub Loading Message". Through the "Advanced options for Ubuntu" menu, select and boot the reinstalled kernel (linux-image-x.x.x-05-generic)
Check if the correct kernel is being used

$ uname -a

You should get similar text: Linux computername x.x.x-05-generic #35-Ubuntu SMP PREEMPT_DYNAMIC Mon Jan 01 00:00:00 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Set CPU governor to "Performance"

$ sudo powerprofilesctl set performance

To get the current active power profile, run the following:

$ sudo powerprofilesctl get

Note: You should get the text "Performance".

Problem: My guest machine freezes consequently my USB audio interface crashing (for passthrough solution using an additional PCIe Gen3 x4 USB 3.2 card).

Action:

Make sure the root directory has approximately 8 GB of free space
Make sure that the Tjunction max value has not been exceeded
Remove unused virtual hardware from your guest machine (Sound, Network, Controllers, Channel, USB Redirection)
In Virtual Machine Manager go to Edit > Connection Details and uncheck 'On Boot' via Virtual Networks tab

Reboot your system
Disable WiFi in Ubuntu
Set CPU governor on Ubuntu operating system to "Performance"

Using the cpupower-gui utility, set the Min freq. value to the base frequency of the processor. Be careful, the processor may overheat (see CPU Tcase)

To use cpupower-gui, install the following package:

$ sudo apt-get install cpupower-gui

Set a "virtual-guest" profile using tuned-gtk

To use tuned-gtk, install the following packages:

$ sudo apt-get install tuned ksmtuned tuned-gtk tuned-utils tuned-utils-systemtap

Optionally install the following, replacing x.x.x-xx with the appropriate version:

$ sudo apt-get install linux-buildinfo-x.x.x-xx-generic linux-cloud-tools-x.x.x-xx linux-cloud-tools-x.x.x-xx-generic linux-cloud-tools-common linux-doc linux-headers-x.x.x-xx linux-headers-x.x.x-xx-generic linux-lib-rust-x.x.x-xx-generic linux-libc-dev linux-modules-x.x.x-xx-generic linux-modules-extra-x.x.x-xx-generic linux-modules-ipu6-x.x.x-xx-generic linux-modules-iwlwifi-x.x.x-xx-generic linux-modules-usbio-x.x.x-xx-generic linux-source-x.x.x linux-tools-x.x.x-xx linux-tools-x.x.x-xx-generic linux-tools-common linux-tools-host

Note: I noticed that the tuned works better after installing the above packages. See Source Package linux.

You can also experimentally run the following commands

Тo launch tuned with profile "virtual-guest", run:

$ sudo powerprofilesctl set performance && sudo service tuned start && sudo systemctl enable --now tuned && sudo tuned-adm profile virtual-guest && sudo tuned -d && sudo tuned-adm active && sudo tuned-adm verify

Тo change the profile, use one of the three lines

$ sudo tuned-adm profile realtime-virtual-guest && sudo tuned-adm active && sudo tuned-adm verify
$ sudo tuned-adm profile virtual-guest && sudo tuned-adm active && sudo tuned-adm verify
$ sudo tuned-adm profile default && sudo tuned-adm active && sudo tuned-adm verify

To permanently disable tuned and revert all changes it performed, run:

$ sudo service tuned stop && sudo tuned-adm off && sudo systemctl disable tuned && sudo killall tuned && sudo powerprofilesctl set balanced && sudo powerprofilesctl list

For Ubuntu 23.04, 23.10

Run the following commands

$ virt-manager
$ htop -t -F libvirtd

The result should look something like:

  [Main] [I/O]
    PID△USER       PRI  NI  VIRT   RES   SHR S  CPU% MEM%   TIME+  Command
    976 root        20   0 1602M 36044 22348 S   0.0  0.2  0:01.32 ├─ /usr/sbin/libvirtd --timeout 120
    .
    .
    999 root        20   0 1602M 36044 22348 S   0.0  0.2  0:00.00 │  ├─ /usr/sbin/libvirtd --timeout 120
   1001 root        20   0 1602M 36044 22348 S   0.0  0.2  0:00.00 │  └─ /usr/sbin/libvirtd --timeout 120
   5905 username    20   0  140M  132M  3584 R  50.0  0.0  0:00.10 │  │     └─ htop -t -F libvird

Run the following commands with the lowest PID value for libvirtd

$ sudo renice -20 -g <PID>
$ sudo chrt -a -r -p 99 <PID>

Run the following command

$ htop -t -F libvirtd

The end result should look something like:

  [Main] [I/O]
    PID△USER       PRI  NI  VIRT   RES   SHR S  CPU% MEM%   TIME+  Command
    976 root        RT -20 1602M 36044 22348 S   0.0  0.2  0:01.32 ├─ /usr/sbin/libvirtd --timeout 120
    .
    .
    999 root        RT -20 1602M 36044 22348 S   0.0  0.2  0:00.00 │  ├─ /usr/sbin/libvirtd --timeout 120
   1001 root        RT -20 1602M 36044 22348 S   0.0  0.2  0:00.00 │  └─ /usr/sbin/libvirtd --timeout 120
   5905 username    20   0  140M  132M  3584 R  50.0  0.0  0:00.10 │  │     └─ htop -t -F libvird

Run your guest machine

For Ubuntu 22.04 LTS, 22.10

Note: Perhaps after Ubuntu 22.04 LTS, 22.10 update the process will become similar to the process for Ubuntu 23.04 or 23.10.

Run your guest machine
Run the following command

$ htop -t

The result should look something like:

  [Main] [I/O]
    PID▽USER       PRI  NI  VIRT   RES   SHR S  CPU% MEM%   TIME+  Command
      2 root        20   0     0     0     0 S   0.0  0.0  0:00.00 kthreadd
  10011 root        20   0     0     0     0 S   0.0  0.0  0:01.17 ├─ 9998
  10005 root        20   0     0     0     0 S   0.0  0.0  0:00.11 ├─ kvm-nx-lpage-recovery-9998
  10004 root         0 -20     0     0     0 I   0.0  0.0  0:00.00 └─ kvm
      1 root        20   0  166M 13300  7344 S   0.0  0.1  0:04.02 init
   9998 libvirt-qe  20   0 11.1G 8371M 11796 S  93.0 52.7  2h35:34 ├─ qemu-system-x86_64 -name guest=username,debug-threads=on -S -object {"qom-type":"secret","
  18086 libvirt-qe  20   0 11.1G 8371M 11796 S   0.0 52.7  0:00.00 │  ├─ worker
  10012 libvirt-qe  20   0 11.1G 8371M 11796 S   0.0 52.7  0:20.20 │  ├─ SPICE Worker
  10010 libvirt-qe  20   0 11.1G 8371M 11796 S   9.3 52.7 18:42.97 │  ├─ CPU 3/KVM
  10009 libvirt-qe  20   0 11.1G 8371M 11796 S  27.9 52.7 18:31.20 │  ├─ CPU 2/KVM
  10008 libvirt-qe  20   0 11.1G 8371M 11796 R  18.6 52.7 25:01.04 │  ├─ CPU 1/KVM
  10007 libvirt-qe  20   0 11.1G 8371M 11796 R   9.3 52.7  1h08:06 │  ├─ CPU 0/KVM
  10006 libvirt-qe  20   0 11.1G 8371M 11796 S   0.0 52.7  0:25.50 │  ├─ IO mon_iothread
  10002 libvirt-qe  20   0 11.1G 8371M 11796 S   0.0 52.7  0:00.11 │  └─ qemu-system-x86_64 -name guest=username,debug-threads=on -S -object {"qom-type":"secret
   9990 root        20   0 80140 13292 11496 S   0.0  0.1  0:00.00 ├─ virtlogd
   1644 username    20   0 19456 10288  7352 S   0.0  0.1  0:00.58 ├─ systemd --user
   9903 username    20   0  937M 89440 39952 S   0.0  0.5  0:40.61 │  └─ python3 /usr/bin/virt-manager
   9908 username    20   0  937M 89440 39952 S   0.0  0.5  0:07.74 │     ├─ virt-manager
   9907 username    20   0  937M 89440 39952 S   0.0  0.5  0:00.00 │     ├─ dconf worker
   9906 username    20   0  937M 89440 39952 S   0.0  0.5  0:00.00 │     ├─ gdbus
   9904 username    20   0  937M 89440 39952 S   0.0  0.5  0:00.00 │     └─ gmain
    960 root        20   0 1526M 34408 19836 S   0.0  0.2  0:22.20 └─ libvirtd
  10001 root        20   0 1526M 34408 19836 S   0.0  0.2  0:08.64    ├─ vm-username
   1014 root        20   0 1526M 34408 19836 S   0.0  0.2  0:00.06    ├─ udev-event
    990 root        20   0 1526M 34408 19836 S   0.0  0.2  0:00.00    ├─ gdbus
    989 root        20   0 1526M 34408 19836 S   0.0  0.2  0:00.00    ├─ gmain
    986 root        20   0 1526M 34408 19836 S   0.0  0.2  0:00.00    ├─ rpc-admin
    985 root        20   0 1526M 34408 19836 S   0.0  0.2  0:00.00    ├─ rpc-admin
    984 root        20   0 1526M 34408 19836 S   0.0  0.2  0:00.00    ├─ rpc-admin
    983 root        20   0 1526M 34408 19836 S   0.0  0.2  0:00.00    ├─ rpc-admin
    982 root        20   0 1526M 34408 19836 S   0.0  0.2  0:00.00    ├─ rpc-admin
    975 root        20   0 1526M 34408 19836 S   0.0  0.2  0:00.22    ├─ prio-rpc-libvir
    974 root        20   0 1526M 34408 19836 S   0.0  0.2  0:00.22    ├─ prio-rpc-libvir
    973 root        20   0 1526M 34408 19836 S   0.0  0.2  0:00.23    ├─ prio-rpc-libvir
    972 root        20   0 1526M 34408 19836 S   0.0  0.2  0:00.21    ├─ prio-rpc-libvir
    971 root        20   0 1526M 34408 19836 S   0.0  0.2  0:00.23    ├─ prio-rpc-libvir
    970 root        20   0 1526M 34408 19836 S   0.0  0.2  0:02.15    ├─ rpc-libvirtd
    969 root        20   0 1526M 34408 19836 S   0.0  0.2  0:02.11    ├─ rpc-libvirtd
    968 root        20   0 1526M 34408 19836 S   0.0  0.2  0:02.19    ├─ rpc-libvirtd
    967 root        20   0 1526M 34408 19836 S   0.0  0.2  0:02.09    ├─ rpc-libvirtd
    966 root        20   0 1526M 34408 19836 S   0.0  0.2  0:02.04    └─ rpc-libvirtd

Set NI (nice) value -5 for all root processes (You will need this every time you start or reboot your operating system)

$ sudo renice -5 -u root

Set NI (nice) value -20 for qemu-system-x86_64 (9998), virt-manager, libvirtd, kvm-nx-lpage-recovery-****, kvm, virtlogd commands (You will need this every time you start your guest machine)

display all the running threads for commands qemu-system-x86_64 (9998), virt-manager, libvirtd

example: $ sudo ps -Ljf <PID>
$ ps -Ljf 9998 9903 960

for commands qemu-system-x86_64 (9998), virt-manager, libvirtd run:

example: $ sudo renice -20 -g <PGID>
$ sudo renice -20 -g 9997 9903 960

for commands kvm-nx-lpage-recovery-9998, kvm, virtlogd run:

example: $ sudo renice -20 <PID>
$ sudo renice -20 10005 10004 9990

Set the commands kvm-nx-lpage-recovery-9998, kvm, qemu-system-x86_64 (9998), virtlogd, virt-manager, libvirtd to real-time (You will need this every time you start your guest machine)

$ sudo chrt -a -r -p 99 10005
$ sudo chrt -a -r -p 99 10004
$ sudo chrt -a -r -p 99 9998
$ sudo chrt -a -r -p 99 9990
$ sudo chrt -a -r -p 99 9903
$ sudo chrt -a -r -p 99 960

Run htop

$ htop -t

The end result should look something like:

  [Main] [I/O]
    PID▽USER       PRI  NI  VIRT   RES   SHR S  CPU% MEM%   TIME+  Command
      2 root        20  -5     0     0     0 S   0.0  0.0  0:00.00 kthreadd
  10011 root        RT -20     0     0     0 S   0.0  0.0  0:01.17 ├─ 9998
  10005 root        RT -20     0     0     0 S   0.0  0.0  0:00.11 ├─ kvm-nx-lpage-recovery-9998
  10004 root        RT -20     0     0     0 I   0.0  0.0  0:00.00 └─ kvm
      1 root        20  -5  166M 13300  7344 S   0.0  0.1  0:04.02 init
   9998 libvirt-qe  RT -20 11.1G 8371M 11796 S  93.0 52.7  2h35:34 ├─ qemu-system-x86_64 -name guest=username,debug-threads=on -S -object {"qom-type":"secret","
  18086 libvirt-qe  RT -20 11.1G 8371M 11796 S   0.0 52.7  0:00.00 │  ├─ worker
  10012 libvirt-qe  RT -20 11.1G 8371M 11796 S   0.0 52.7  0:20.20 │  ├─ SPICE Worker
  10010 libvirt-qe  RT -20 11.1G 8371M 11796 S   9.3 52.7 18:42.97 │  ├─ CPU 3/KVM
  10009 libvirt-qe  RT -20 11.1G 8371M 11796 S  27.9 52.7 18:31.20 │  ├─ CPU 2/KVM
  10008 libvirt-qe  RT -20 11.1G 8371M 11796 R  18.6 52.7 25:01.04 │  ├─ CPU 1/KVM
  10007 libvirt-qe  RT -20 11.1G 8371M 11796 R   9.3 52.7  1h08:06 │  ├─ CPU 0/KVM
  10006 libvirt-qe  RT -20 11.1G 8371M 11796 S   0.0 52.7  0:25.50 │  ├─ IO mon_iothread
  10002 libvirt-qe  RT -20 11.1G 8371M 11796 S   0.0 52.7  0:00.11 │  └─ qemu-system-x86_64 -name guest=username,debug-threads=on -S -object {"qom-type":"secret
   9990 root        RT -20 80140 13292 11496 S   0.0  0.1  0:00.00 ├─ virtlogd
   1644 username    20   0 19456 10288  7352 S   0.0  0.1  0:00.58 ├─ systemd --user
   9903 username    RT -20  937M 89440 39952 S   0.0  0.5  0:40.61 │  └─ python3 /usr/bin/virt-manager
   9908 username    RT -20  937M 89440 39952 S   0.0  0.5  0:07.74 │     ├─ virt-manager
   9907 username    RT -20  937M 89440 39952 S   0.0  0.5  0:00.00 │     ├─ dconf worker
   9906 username    RT -20  937M 89440 39952 S   0.0  0.5  0:00.00 │     ├─ gdbus
   9904 username    RT -20  937M 89440 39952 S   0.0  0.5  0:00.00 │     └─ gmain
    960 root        RT -20 1526M 34408 19836 S   0.0  0.2  0:22.20 └─ libvirtd
  10001 root        RT -20 1526M 34408 19836 S   0.0  0.2  0:08.64    ├─ vm-username
   1014 root        RT -20 1526M 34408 19836 S   0.0  0.2  0:00.06    ├─ udev-event
    990 root        RT -20 1526M 34408 19836 S   0.0  0.2  0:00.00    ├─ gdbus
    989 root        RT -20 1526M 34408 19836 S   0.0  0.2  0:00.00    ├─ gmain
    986 root        RT -20 1526M 34408 19836 S   0.0  0.2  0:00.00    ├─ rpc-admin
    985 root        RT -20 1526M 34408 19836 S   0.0  0.2  0:00.00    ├─ rpc-admin
    984 root        RT -20 1526M 34408 19836 S   0.0  0.2  0:00.00    ├─ rpc-admin
    983 root        RT -20 1526M 34408 19836 S   0.0  0.2  0:00.00    ├─ rpc-admin
    982 root        RT -20 1526M 34408 19836 S   0.0  0.2  0:00.00    ├─ rpc-admin
    975 root        RT -20 1526M 34408 19836 S   0.0  0.2  0:00.22    ├─ prio-rpc-libvir
    974 root        RT -20 1526M 34408 19836 S   0.0  0.2  0:00.22    ├─ prio-rpc-libvir
    973 root        RT -20 1526M 34408 19836 S   0.0  0.2  0:00.23    ├─ prio-rpc-libvir
    972 root        RT -20 1526M 34408 19836 S   0.0  0.2  0:00.21    ├─ prio-rpc-libvir
    971 root        RT -20 1526M 34408 19836 S   0.0  0.2  0:00.23    ├─ prio-rpc-libvir
    970 root        RT -20 1526M 34408 19836 S   0.0  0.2  0:02.15    ├─ rpc-libvirtd
    969 root        RT -20 1526M 34408 19836 S   0.0  0.2  0:02.11    ├─ rpc-libvirtd
    968 root        RT -20 1526M 34408 19836 S   0.0  0.2  0:02.19    ├─ rpc-libvirtd
    967 root        RT -20 1526M 34408 19836 S   0.0  0.2  0:02.09    ├─ rpc-libvirtd
    966 root        RT -20 1526M 34408 19836 S   0.0  0.2  0:02.04    └─ rpc-libvirtd

Use 'q' to stop

If the issue is not completely resolved try to use fewer CPU cores. Referring to physical CPU topology set the number of cores to half. For example, the Intel® Core™ i5-6600 Processor has 4 cores. To use half number of cores set CPU topology to 1 sockets, 2 cores, 1 threads.
Increase the buffer size for your audio interface's ASIO driver.

Problem: My guest machine shuts down suddenly.

Action:

To display and monitor hardware temperature, install the following packages

$ sudo apt install lm-sensors glances psensor

Note: Run sudo sensors-detect before running sensors.

If the CPU is overheating, check whether overclocking mode is enabled in the BIOS. If yes, turn it off. To learn about the high and critical temperature of the CPU, refer to its datasheet.

Note: You can also find out about high and critical CPU temperature using "sensors" tool.

Try to set the profile to "Balanced" or "Power Saver"

Note: If the problem persists, contact a technician for advanced system diagnostics. It may be necessary to replace the thermal paste or the CPU cooling system.

Problem: Heavy load on the CPU when accessing the qcow2 disk image (Virtual Disk). Intermittent sound with crackling on virtual machine using sample libraries.

Action:

set virtual disk cache mode to unsafe
passthrough an additional physical SSD M.2 or passthrough an additional SATA 3.0 PCIe Gen3 x1 card for SATA SSD/HDD.

Problem: USB devices (HDD/SSD, Webcam, Midi keyboard) stop responding after a few minutes inactivity on a Windows 10 system.

Action:

disable USB Selective Suspending via Power Options

prevent the Windows 10 OS to turn off Generic SuperSpeed USB Hub, Generic USB Hub, USB Root Hub devices to save power via Device Manager.

Ubuntu 22.04 GPU passthrough (QEMU)

3 Answers3

GPU PASSTHROUGH GUIDE FOR UBUNTU WITH NOUVEAU DRIVER INSTALLED

HARDWARE SPECIFICATION REQUIRED FOR THIS GUIDE

REQUIRED STEPS

NOTE

SEE ALSO

TROUBLESHOOTING

REFERENCES

GPU PASSTHROUGH GUIDE FOR UBUNTU WITH NVIDIA PROPRIETARY DRIVER INSTALLED

HARDWARE SPECIFICATION REQUIRED FOR THIS GUIDE

REQUIRED STEPS

NOTE

SEE ALSO

TROUBLESHOOTING

REFERENCES

GPU PASSTHROUGH TROUBLESHOOTING

REFERENCES

Linked