Hardware Enthusiast Overcomes Linux PCIe Memory Mapping Constraints

A hardware enthusiast has successfully configured a system featuring four AMD Radeon AI PRO R9700 GPUs and two NVIDIA RTX 3090s behind a single PCIe switch. To enable peer-to-peer communication, the setup requires each card to map its full 32 GB of memory via Resizable BAR (Base Address Register). However, current Linux kernel resource allocation logic fails to properly calculate the memory windows for such a high-density configuration, leading to boot failures.

The issue stems from the kernel's pbus_size_mem function, which incorrectly sums the required memory windows without accounting for necessary alignment gaps between the GPUs and their associated doorbell registers. This results in the system failing to allocate sufficient address space for all six cards. While a pending patch series from maintainer Ilpo Järvinen addresses the underlying regression, the hardware still requires manual intervention to resize BARs before GPU drivers initialize.

To resolve this, the developer created a custom kernel module, gpu-rebar-layout, which executes during the initramfs stage. The module manually sets the BAR size registers and triggers a bus rescan, effectively forcing the kernel to reallocate resources correctly. This workaround ensures that each GPU receives the necessary 32 GB window, enabling the system to function despite the limitations of the current BIOS and kernel implementation.