PCAT BIOS SMP AP-startup rendezvous race deadlocks boot under virtualization

[smalis-msft]

Note: This issue was written and filed by an AI agent (GitHub Copilot) on behalf of @smalis-msft.

Summary

The proprietary Hyper-V PCAT BIOS has a latent SMP application-processor (AP) startup race that can deadlock guest boot under virtualization. It surfaces as an intermittent boot_heavy (16-VP) timeout on PCAT configs — e.g. multiarch::hyperv_openhcl_pcat_x64_windows_datacenter_core_2022_x64_boot_heavy in run 28832879807. This appears to be the underlying root cause behind #1684 (and the "OpenVMM PCAT has some sort of race condition" line item on #2139).

Symptom

• 10-minute test timeout, stuck at "Waiting for boot event…", all-black screenshots.
• VTL2 OpenHCL boots fine (~13s), then no VTL0 progress until timeout diagnostics.
• timeout_inspect_openhcl.log: ~13 VPs mp_state=wait_for_sipi; BSP in real mode (cr0 PE=0, cs.base=0xf0000) spinning cmp word ptr gs:[10h],0 / jne; one AP spinning lock bts word ptr gs:[10h],0 / jb; one AP parked hlt. All 16 VPs briefly ran the AP trampoline (guest tried to set high-bit in CMOS addr register) before the hang.

Root cause (disassembled from vmfirmwarepcat.dll, resource VMFW/13500)

The BIOS brings APs up one at a time using a spinlock word at gs:[0x10] (guest-linear 0x30010):

BSP, per AP:                       AP, on SIPI:
  call send_ipi   ; send SIPI        lock bts word gs:[10h],0   ; acquire
  cmp  word gs:[10h],0               ... init ...
  je   next_ap    ; <-- BUG          lock btr word gs:[10h],0   ; release
  spin: cmp word gs:[10h],0          hlt
        jne spin  ; wait release
  call send_ipi   ; INIT -> park the AP
next_ap:

(send_ipi writes the local APIC ICR at [apicbase+0x310]/[apicbase+0x300]; SIPI = esi=0x4600|vector, INIT = esi=0x500.)

The je next_ap short-circuit assumes the AP has already grabbed the lock by the time the BSP checks it — i.e. that the AP starts executing within the BSP's fixed post-SIPI delay. On real hardware an AP starts in microseconds, so this always holds. Under virtualization the SIPI → AP-execution latency (host vCPU scheduling plus the AP trampoline's emulated CMOS/port exits) can exceed that delay. When it does, the BSP desyncs by one AP and sends its park-INIT to an AP that currently holds the lock, resetting it mid-critical-section. The lock word is then stuck set and the BSP + the next AP spin forever.

This reproduces the captured deadlock exactly: the lock-holder is reset back into wait_for_sipi, the BSP waits for a release that never comes, and the next AP can never acquire.

Why this can't be fixed from this repo (for the Hyper-V-backed config)

Every element of the race is outside OpenVMM's control:

• BIOS is a proprietary blob (Microsoft.Windows.VmFirmware.Pcat / host vmfirmwarepcat.dll) — no source to recompile.
• PCAT ⇒ Hyper-V Generation 1 VM, so the host worker (vmwp.exe) loads the BIOS ROM; OpenVMM/OpenHCL never map those bytes → can't binary-patch them.
• Non-isolated VTL0 APIC + INIT/SIPI delivery is hypervisor-owned → can't reduce SIPI-to-AP latency.
• The post-SIPI delay is a fixed CPU spin in the ROM.

(For the OpenVMM software-APIC backend we do map the ROM and could in principle patch the in-memory copy, but that's fragile proprietary-firmware hacking and wouldn't help the Hyper-V test.)

Current mitigation

petri flaky_boot VmmQuirks: reset the VM and retry the boot if no boot event arrives within 15s (good case is < 1s). This was OpenVMM-backend-only; it is now also applied to the Hyper-V backend for PCAT firmware (petri/src/vm/hyperv/mod.rs).

References

• Repro: run 28832879807, test multiarch::hyperv_openhcl_pcat_x64_windows_datacenter_core_2022_x64_boot_heavy
• Related: flaky test: openvmm pcat stuck at cursor #1684 (closed), VMM Test Flakiness Master List #2139 (VMM test flakiness master list)

---

Filed by GitHub Copilot (AI agent) at the request of @smalis-msft.