Skip to content

refactor(aarch64): migrate virtual IPIs off PERCPU_EVENTS - #407

Merged
li041 merged 9 commits into
syswonder:devfrom
peikachu:refactor/remove-percpu-events
Oct 30, 2026
Merged

li041 merged 9 commits into
syswonder:devfrom
peikachu:refactor/remove-percpu-events

Conversation

@peikachu

Copy link
Copy Markdown
Collaborator

Summary

This pull request starts the staged removal of PERCPU_EVENTS by migrating AArch64 virtual IPIs to per-vCPU interrupt state.

  • Add a common GuestCpuIrqState pending bitmap and architecture-private ArchVcpu state.
  • Post GICv2 and GICv3 guest SGIs to the target vCPU, then flush them locally into list registers.
  • Preserve GICv2 SGI source information and virtualize the relevant pending-register behavior.
  • Flush posted interrupts from host-doorbell, maintenance, and guest-entry paths while the legacy event FIFO remains available for unmigrated event classes.
  • Reserve physical SGI 7 for the host doorbell and stop treating other physical SGIs as guest virtual IPIs.
  • Resolve the existing virtio-pci event ID collisions by moving CONFIG, DATA, and DONE to IDs 10-12.

Motivation

Virtual IPIs describe guest interrupt state and do not require FIFO payload semantics. Routing them through PERCPU_EVENTS adds locking, possible queue allocation, and unnecessary coupling between guest interrupt delivery and host CPU event dispatch.

The new path publishes pending state before kicking the target CPU. Only the target CPU installs the interrupt into its local GIC list registers. This keeps interrupt state with the guest CPU, supports the current pinned pCPU:vCPU mapping, and provides a foundation for removing additional event categories without introducing a scheduler in this change.

The event ID update also removes collisions between virtio-pci events, vCPU suspend/resume events, and DWC MSI injection.

Verification

  • On AArch64 QEMU with the qemu-gicv3 configuration, zone1 booted successfully and reached its shell.

Risks and Limitations

  • Only AArch64 virtual IPIs are migrated. PPI, SPI, LPI, DWC MSI, virtio IRQ, and other architecture event classes still use their existing paths.
  • The legacy PERCPU_EVENTS FIFO and GIC PENDING_VIRQS queue remain for unmigrated producers.
  • GICv2 QEMU reached a running zone1 with both assigned CPUs started, but the zone1 virtio console did not produce output, so a complete GICv2 zone1 shell regression is still pending.
  • Cross-CPU pending clears remove software pending state but do not retract an interrupt that has already been claimed or installed in a remote CPU's list register.
  • The implementation retains the current pinned pCPU:vCPU mapping and does not add vCPU scheduling or migration.

Comment thread src/cpu_data.rs Resolved
Comment thread src/device/irqchip/gicv2/gic.rs Resolved
Comment thread src/device/irqchip/gicv2/gic.rs Resolved
Comment thread src/device/irqchip/gicv2/vgic.rs Resolved
@li041

li041 commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

Thanks for the contribution! This is a substantial refactor and the migration of AArch64 virtual IPIs off PERCPU_EVENTS is clearly a step in the right direction. That said, I think the existing implementation may need to be rethought and cleaned up more broadly, and this PR looks like a good opportunity to do that.

Also, this change seems to introduce some bugs. It may take some effort to resolve them.

Comment thread src/device/irqchip/gicv2/vgic.rs Resolved
Comment thread src/device/irqchip/gicv2/vgic.rs Resolved
Comment thread src/device/irqchip/gicv2/gic.rs Resolved
Comment thread src/device/irqchip/gicv3/mod.rs Resolved
Comment thread src/device/irqchip/gicv3/mod.rs Resolved
@peikachu
peikachu force-pushed the refactor/remove-percpu-events branch from d287afe to a27b87f Compare October 26, 2026 08:53
Comment thread src/arch/aarch64/trap.rs Resolved
Comment thread src/arch/aarch64/vcpu.rs Resolved
Comment thread src/arch/loongarch64/cpu.rs Resolved
Comment thread src/arch/riscv64/trap.rs Resolved
Comment thread src/cpu_data.rs Resolved
Comment thread src/device/irqchip/gicv3/mod.rs Resolved
Comment thread src/cpu_data.rs Resolved
Comment thread src/device/irqchip/gicv2/gic.rs Resolved
Comment thread src/event.rs Resolved
Comment thread src/device/irqchip/gicv2/gich.rs Resolved
Comment thread src/device/irqchip/gicv2/gich.rs Resolved

@li041 li041 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for consistently addressing the review feedback throughout this PR. The latest revisions have significantly improved the abstraction boundaries and cleaned up the ARM interrupt-controller code.

There are still some limitations in the current vCPU abstraction. For example, each Vcpu remains embedded in and pinned to a single PerCpu; architectural context, virtual timer state, and GICH/ICH state are not yet owned and saved by the vCPU. Therefore, this abstraction is a useful foundation, but it does not yet support scheduling or time-sharing multiple vCPUs on one pCPU.

The existing GICv2 and GICv3 implementations also contain several pre-existing issues that are outside the scope of this PR. Since this change substantially modifies the SGI delivery and pending-state paths, please run the relevant GICv2 and GICv3 boot and workload tests multiple times, especially with multi zones, to make sure the result is stable and does not introduce intermittent interrupt loss or startup failures.

@peikachu
peikachu force-pushed the refactor/remove-percpu-events branch from d28bd9f to 3fcb5a7 Compare October 29, 2026 07:16
@peikachu
peikachu force-pushed the refactor/remove-percpu-events branch from 3fcb5a7 to ddb134f Compare October 29, 2026 07:43
@peikachu

Copy link
Copy Markdown
Collaborator Author

Reorganized the commits by feature just now (the final code is unchanged).

@li041
li041 merged commit a11f2fd into syswonder:dev Oct 30, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants