[design] Lowering hand-built generic MIR (Route A) blocked by GlobalISel/SelectionDAG entry-point tension
- Dominant language
- Python
- Stars
- 79
- Forks
- 14
- Avg merge
- 11h 43m
- Merged PRs (30d)
- 72
Description
## Lowering hand-built *generic* MIR (Route A) is blocked by the GlobalISel↔SelectionDAG entry-point tension
The MIR stack (#579–#591) ended up supporting two routes from Python-built MIR to native code. **Route B** is implemented; **Route A** is not, and this issue records *why* — the reason is architectural, not an oversight, and it's worth having written down.
### The two routes
- **Route A — build *generic* (`G_*`) MIR, then run instruction selection on it.**
`@machine_function` / `MachineIRBuilder` produce target-independent `G_ADD`/`G_ICMP`/`G_PHI`/… MIR. The natural next step would be to hand that MIR to the target's post-IRTranslator pipeline (Legalizer → RegBankSelect → InstructionSelect → regalloc → emission) and get an object out. **This does not work today.**
- **Route B — build *already-selected* target MIR by hand, run only the back half of codegen.** (#589, #590)
Resolve real register classes / physregs, emit target opcodes (`ADDWrr`, `COPY`, `RET_ReallyLR`), set the `MachineFunctionProperties`, then `emit_object()` runs `addPassesToEmitFile` with `-start-after=finalize-isel` so **no instruction selection runs at all** — just regalloc + prologue/epilogue + emission. This is what the capstone JIT-executes. It sidesteps the entire problem below.
### Why Route A is blocked
1. **SelectionDAG is not a MIR→MIR pass.** `SelectionDAGISel` builds its DAG *from IR* (one basic block at a time) and only then emits MIR. So the SelectionDAG path fundamentally cannot *consume* hand-built generic MIR — there is nothing to feed it but IR.
2. **GlobalISel *is* MIR→MIR, but its failure path re-enters SelectionDAG *from IR*.** GlobalISel's `InstructionSelect` (and the earlier legalize/regbankselect steps) operate on MIR, so in principle they could lower hand-built generic MIR. But when GlobalISel can't handle a construct, the abort mode falls back to SelectionDAG — which needs the IR. Hand-built MIR has no faithful IR (Route B's `create_machine_function` attaches only a trivial stub definition so codegen doesn't skip the function), so the fallback can't reproduce the intended semantics. `run_codegen_to_mir(global_isel=True)` (#588) works precisely because it starts *from IR*: it runs IRTranslator → Legalizer → RegBankSelect → InstructionSelect and never has externally-authored generic MIR to preserve.
3. **The pipeline entry point is IR-shaped.** `TargetPassConfig` / `addPassesToEmitFile` build a pipeline that begins at the IR. To lower *pre-existing* generic MIR you'd need to start it partway (e.g. `-start-before=legalizer` / after IRTranslator) while feeding in the already-built `MachineFunction`s. The only levers today are the fragile process-global `start-before`/`start-after` `cl::opt`s (Route B uses `start-after=finalize-isel` with a set/restore guard — see #590 and the note in #593 item 1) or a programmatic `TargetPassConfig::setStartStopPasses`, which isn't cleanly bound.
### Current mitigations already in place (context, not fixes for Route A)
- #588: `run_codegen_to_mir(global_isel=True)` uses `GlobalISelAbortMode::DisableWithDiag` (aborting mode would kill the process) and then **scans the result for residual `G_*` ops**, raising if selection didn't complete — because the fallback's diagnostic is only a warning. It also sets the GlobalISel flags per-call (AArch64 defaults `EnableGlobalISel=true`).
- These make the *IR→selected-MIR* path safe; they do **not** enable *hand-built-generic-MIR→selected-MIR*.
### What would unblock Route A (for discussion)
- A clean binding for `TargetPassConfig::setStartStopPasses` (or equivalent) to construct a "start at Legalizer, stop after emission" pipeline over externally-built generic MIR — replacing the process-global `start-*` `cl::opt` mechanism entirely.
- Running the GlobalISel legalize→select passes on hand-built MIR with the SelectionDAG fallback **disabled** and unselected ops surfaced as a catchable error (the residual-`G_*` scan from #588 is half of this already).
- Confirming the `MachineModuleInfoWrapperPass` build path (from `create_machine_function`) can carry generic MIR through those passes, and deciding what — if any — IR stub the functions need for the passes that still consult it.
Until one of those lands, **Route B is the supported way to reach native code from Python-built MIR**, and Route A generic-MIR lowering is only reachable indirectly by going back through IR (`run_codegen_to_mir`).
---
*Refs: #588 (GlobalISel gating + residual scan), #589 (build selected MIR), #590 (`emit_object` / `-start-after=finalize-isel`), and the deferred-items tracker #593 (item 1 covers the `start-after` `cl::opt` fragility).*
Contributor guide
No contributing guide indexed for this repository
Research direction
Read the Route A discussion alongside #588, #590, and #593, then inspect run_codegen_to_mir, create_machine_function, emit_object, and TargetPassConfig::setStartStopPasses. The work would need a defined pipeline for externally built generic MIR, disabled SelectionDAG fallback with catchable failures, and confirmation of the MachineModuleInfoWrapperPass and IR-stub behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- compilers
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100