GPU Passthrough

HYPR supports GPU passthrough for running GPU-accelerated workloads in VMs.

Platform Support

Platform	Method	Supported GPUs
Linux	VFIO	NVIDIA, AMD, Intel
macOS Apple Silicon	Metal via Venus	Integrated GPU
macOS Intel	Not supported	-

Linux: VFIO Passthrough

On Linux, GPUs are passed through using VFIO (Virtual Function I/O), which provides direct hardware access to the VM.

Prerequisites

IOMMU Enabled: Add to kernel command line:
- Intel: intel_iommu=on
- AMD: amd_iommu=on
VFIO Modules: Ensure modules are loaded:
sudo modprobe vfio-pci

List Available GPUs

hypr gpu list

Output:

PCI ADDRESS     VENDOR     MODEL                          DRIVER       IOMMU      STATUS
0000:01:00.0    Nvidia     NVIDIA GeForce RTX 4090        nvidia       1          available
0000:02:00.0    Amd        AMD Radeon RX 7900 XTX         amdgpu       2          available

Status values:

available - GPU can be used for passthrough
vfio-ready - GPU is already bound to vfio-pci driver
boot-vga - GPU is the primary display (cannot unbind without —force)

Run with GPU

hypr run --gpu 0000:01:00.0 pytorch/pytorch:latest

This:

Validates the PCI address
Checks IOMMU group isolation
Binds the GPU to vfio-pci driver
Passes the GPU to the VM
Restores the original driver when VM stops

IOMMU Groups

All devices in an IOMMU group must be passed through together. HYPR validates this automatically. If validation fails:

GPU 0000:01:00.0 not in isolated IOMMU group.
Other devices in group: 0000:01:00.1 (audio)

Pass all devices in the group:

hypr run --gpu 0000:01:00.0 --gpu 0000:01:00.1 pytorch

Boot VGA Protection

The boot VGA device (primary GPU driving the display) is protected by default. Unbinding it can hang the system. To override (use with caution):

# Only if you have another GPU or headless setup
hypr run --gpu 0000:00:02.0 --force-gpu pytorch

NVIDIA-Specific Notes

Driver Installation in VM: The VM needs NVIDIA drivers. Use an image with drivers pre-installed:

hypr run --gpu 0000:01:00.0 nvidia/cuda:12.0-runtime-ubuntu22.04

SR-IOV (Virtual Functions): Some enterprise GPUs (A100, H100) support SR-IOV for sharing a GPU across multiple VMs. HYPR will detect and use virtual functions when available.

macOS: Metal GPU

On Apple Silicon Macs, HYPR uses Metal GPU virtualization via the Venus Vulkan driver.

How It Works

libkrun creates a VM with virtio-gpu device
Guest uses Venus driver (Vulkan over virtio-gpu)
Host translates Vulkan to Metal
GPU operations run on Apple GPU

Enable Metal GPU

hypr run --gpu tensorflow/tensorflow:latest-gpu

No PCI address needed - the integrated GPU is used automatically.

List GPU

hypr gpu list

Output:

VENDOR     MODEL                                    MEMORY          STATUS
Apple      Apple M3 Max                             48.0 GB         available

Performance

Metal passthrough achieves approximately 60-80% of native performance, depending on workload:

Compute (matrix ops): ~75-80%
Graphics rendering: ~60-70%
Memory bandwidth: ~70-75%

Supported Frameworks

Frameworks using Metal Compute or MPS work automatically:

TensorFlow with Metal plugin
PyTorch with MPS backend
MLX
Core ML

Troubleshooting

Linux: GPU Not Available

IOMMU not enabled:

dmesg | grep -i iommu

Should show IOMMU initialization. If not, check kernel command line. VFIO module not loaded:

lsmod | grep vfio

Load manually: sudo modprobe vfio-pci GPU in use by host: Stop display manager and unload GPU driver:

sudo systemctl stop gdm
sudo rmmod nvidia_drm nvidia_modeset nvidia

macOS: Metal Not Working

Check libkrun-efi version:

brew info libkrun-efi

Requires libkrun-efi with Metal support. Verify macOS version: Metal GPU requires macOS 14 (Sonoma) or later. Check VM configuration: GPU must be enabled at VM creation. Cannot be hot-added.

Check GPU Usage in VM

NVIDIA (Linux):

hypr exec myvm -- nvidia-smi

AMD (Linux):

hypr exec myvm -- rocm-smi

Metal (macOS guest):

hypr exec myvm -- python -c "import torch; print(torch.backends.mps.is_available())"

Compose with GPU

Define GPU access in compose files:

services:
  ml-worker:
    image: pytorch/pytorch:latest
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

On Linux, specify the GPU:

# Deploy with GPU passthrough
hypr compose up

On macOS Apple Silicon, GPU access is automatic when Metal is enabled:

hypr compose up

Best Practices

Use pre-built GPU images with drivers included
Test GPU access before deploying workloads
Monitor GPU memory to avoid OOM errors
Restore drivers after VM shutdown (automatic)
Isolate IOMMU groups for reliable passthrough

Introduction

Core Concepts

References

GPU

GPU Passthrough

Platform Support

Linux: VFIO Passthrough

Prerequisites

List Available GPUs

Run with GPU

IOMMU Groups

Boot VGA Protection

NVIDIA-Specific Notes

macOS: Metal GPU

How It Works

Enable Metal GPU

List GPU

Performance

Supported Frameworks

Troubleshooting

Linux: GPU Not Available

macOS: Metal Not Working

Check GPU Usage in VM

Compose with GPU

Best Practices

​GPU Passthrough

​Platform Support

​Linux: VFIO Passthrough

​Prerequisites

​List Available GPUs

​Run with GPU

​IOMMU Groups

​Boot VGA Protection

​NVIDIA-Specific Notes

​macOS: Metal GPU

​How It Works

​Enable Metal GPU

​List GPU

​Performance

​Supported Frameworks

​Troubleshooting

​Linux: GPU Not Available

​macOS: Metal Not Working

​Check GPU Usage in VM

​Compose with GPU

​Best Practices

GPU Passthrough

Platform Support

Linux: VFIO Passthrough

Prerequisites

List Available GPUs

Run with GPU

IOMMU Groups

Boot VGA Protection

NVIDIA-Specific Notes

macOS: Metal GPU

How It Works

Enable Metal GPU

List GPU

Performance

Supported Frameworks

Troubleshooting

Linux: GPU Not Available

macOS: Metal Not Working

Check GPU Usage in VM

Compose with GPU

Best Practices