Skip to main content

​
GPU Passthrough

HYPR supports GPU passthrough for running GPU-accelerated workloads in VMs.

​
Platform Support

PlatformMethodSupported GPUs
LinuxVFIONVIDIA, AMD, Intel
macOS Apple SiliconMetal via VenusIntegrated GPU
macOS IntelNot supported-

​
Linux: VFIO Passthrough

On Linux, GPUs are passed through using VFIO (Virtual Function I/O), which provides direct hardware access to the VM.

​
Prerequisites

  1. IOMMU Enabled: Add to kernel command line:
    • Intel: intel_iommu=on
    • AMD: amd_iommu=on
  2. VFIO Modules: Ensure modules are loaded: 
    sudo modprobe vfio-pci

​
List Available GPUs

hypr gpu list
Output: 
PCI ADDRESS     VENDOR     MODEL                          DRIVER       IOMMU      STATUS
0000:01:00.0    Nvidia     NVIDIA GeForce RTX 4090        nvidia       1          available
0000:02:00.0    Amd        AMD Radeon RX 7900 XTX         amdgpu       2          available
Status values:
  • available - GPU can be used for passthrough
  • vfio-ready - GPU is already bound to vfio-pci driver
  • boot-vga - GPU is the primary display (cannot unbind without —force)

​
Run with GPU

hypr run --gpu 0000:01:00.0 pytorch/pytorch:latest
This:
  1. Validates the PCI address
  2. Checks IOMMU group isolation
  3. Binds the GPU to vfio-pci driver
  4. Passes the GPU to the VM
  5. Restores the original driver when VM stops

​
IOMMU Groups

All devices in an IOMMU group must be passed through together. HYPR validates this automatically. If validation fails: 
GPU 0000:01:00.0 not in isolated IOMMU group.
Other devices in group: 0000:01:00.1 (audio)
Pass all devices in the group: 
hypr run --gpu 0000:01:00.0 --gpu 0000:01:00.1 pytorch

​
Boot VGA Protection

The boot VGA device (primary GPU driving the display) is protected by default. Unbinding it can hang the system. To override (use with caution): 
# Only if you have another GPU or headless setup
hypr run --gpu 0000:00:02.0 --force-gpu pytorch

​
NVIDIA-Specific Notes

Driver Installation in VM: The VM needs NVIDIA drivers. Use an image with drivers pre-installed: 
hypr run --gpu 0000:01:00.0 nvidia/cuda:12.0-runtime-ubuntu22.04
SR-IOV (Virtual Functions): Some enterprise GPUs (A100, H100) support SR-IOV for sharing a GPU across multiple VMs. HYPR will detect and use virtual functions when available.

​
macOS: Metal GPU

On Apple Silicon Macs, HYPR uses Metal GPU virtualization via the Venus Vulkan driver.

​
How It Works

  1. libkrun creates a VM with virtio-gpu device
  2. Guest uses Venus driver (Vulkan over virtio-gpu)
  3. Host translates Vulkan to Metal
  4. GPU operations run on Apple GPU

​
Enable Metal GPU

hypr run --gpu tensorflow/tensorflow:latest-gpu
No PCI address needed - the integrated GPU is used automatically.

​
List GPU

hypr gpu list
Output: 
VENDOR     MODEL                                    MEMORY          STATUS
Apple      Apple M3 Max                             48.0 GB         available

​
Performance

Metal passthrough achieves approximately 60-80% of native performance, depending on workload:
  • Compute (matrix ops): ~75-80%
  • Graphics rendering: ~60-70%
  • Memory bandwidth: ~70-75%

​
Supported Frameworks

Frameworks using Metal Compute or MPS work automatically:
  • TensorFlow with Metal plugin
  • PyTorch with MPS backend
  • MLX
  • Core ML

​
Troubleshooting

​
Linux: GPU Not Available

IOMMU not enabled: 
dmesg | grep -i iommu
Should show IOMMU initialization. If not, check kernel command line. VFIO module not loaded: 
lsmod | grep vfio
Load manually: sudo modprobe vfio-pci GPU in use by host: Stop display manager and unload GPU driver: 
sudo systemctl stop gdm
sudo rmmod nvidia_drm nvidia_modeset nvidia

​
macOS: Metal Not Working

Check libkrun-efi version: 
brew info libkrun-efi
Requires libkrun-efi with Metal support. Verify macOS version: Metal GPU requires macOS 14 (Sonoma) or later. Check VM configuration: GPU must be enabled at VM creation. Cannot be hot-added.

​
Check GPU Usage in VM

NVIDIA (Linux): 
hypr exec myvm -- nvidia-smi
AMD (Linux): 
hypr exec myvm -- rocm-smi
Metal (macOS guest): 
hypr exec myvm -- python -c "import torch; print(torch.backends.mps.is_available())"

​
Compose with GPU

Define GPU access in compose files: 
services:
  ml-worker:
    image: pytorch/pytorch:latest
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
On Linux, specify the GPU: 
# Deploy with GPU passthrough
hypr compose up
On macOS Apple Silicon, GPU access is automatic when Metal is enabled: 
hypr compose up

​
Best Practices

  1. Use pre-built GPU images with drivers included
  2. Test GPU access before deploying workloads
  3. Monitor GPU memory to avoid OOM errors
  4. Restore drivers after VM shutdown (automatic)
  5. Isolate IOMMU groups for reliable passthrough