llvm / llvm/eudsl

[design] Lowering hand-built generic MIR (Route A) blocked by GlobalISel/SelectionDAG entry-point tension

Open
#594 2 comments 0 reactions 0 assignees View on GitHub
enhancement question
Dominant language
Python
Stars
79
Forks
14
Avg merge
11h 43m
Merged PRs (30d)
72

Description

## Lowering hand-built *generic* MIR (Route A) is blocked by the GlobalISel↔SelectionDAG entry-point tension

The MIR stack (#579–#591) ended up supporting two routes from Python-built MIR to native code. **Route B** is implemented; **Route A** is not, and this issue records *why* — the reason is architectural, not an oversight, and it's worth having written down.

### The two routes

- **Route A — build *generic* (`G_*`) MIR, then run instruction selection on it.**
`@machine_function` / `MachineIRBuilder` produce target-independent `G_ADD`/`G_ICMP`/`G_PHI`/… MIR. The natural next step would be to hand that MIR to the target's post-IRTranslator pipeline (Legalizer → RegBankSelect → InstructionSelect → regalloc → emission) and get an object out. **This does not work today.**

- **Route B — build *already-selected* target MIR by hand, run only the back half of codegen.** (#589, #590)
Resolve real register classes / physregs, emit target opcodes (`ADDWrr`, `COPY`, `RET_ReallyLR`), set the `MachineFunctionProperties`, then `emit_object()` runs `addPassesToEmitFile` with `-start-after=finalize-isel` so **no instruction selection runs at all** — just regalloc + prologue/epilogue + emission. This is what the capstone JIT-executes. It sidesteps the entire problem below.

### Why Route A is blocked

1. **SelectionDAG is not a MIR→MIR pass.** `SelectionDAGISel` builds its DAG *from IR* (one basic block at a time) and only then emits MIR. So the SelectionDAG path fundamentally cannot *consume* hand-built generic MIR — there is nothing to feed it but IR.

2. **GlobalISel *is* MIR→MIR, but its failure path re-enters SelectionDAG *from IR*.** GlobalISel's `InstructionSelect` (and the earlier legalize/regbankselect steps) operate on MIR, so in principle they could lower hand-built generic MIR. But when GlobalISel can't handle a construct, the abort mode falls back to SelectionDAG — which needs the IR. Hand-built MIR has no faithful IR (Route B's `create_machine_function` attaches only a trivial stub definition so codegen doesn't skip the function), so the fallback can't reproduce the intended semantics. `run_codegen_to_mir(global_isel=True)` (#588) works precisely because it starts *from IR*: it runs IRTranslator → Legalizer → RegBankSelect → InstructionSelect and never has externally-authored generic MIR to preserve.

3. **The pipeline entry point is IR-shaped.** `TargetPassConfig` / `addPassesToEmitFile` build a pipeline that begins at the IR. To lower *pre-existing* generic MIR you'd need to start it partway (e.g. `-start-before=legalizer` / after IRTranslator) while feeding in the already-built `MachineFunction`s. The only levers today are the fragile process-global `start-before`/`start-after` `cl::opt`s (Route B uses `start-after=finalize-isel` with a set/restore guard — see #590 and the note in #593 item 1) or a programmatic `TargetPassConfig::setStartStopPasses`, which isn't cleanly bound.

### Current mitigations already in place (context, not fixes for Route A)

- #588: `run_codegen_to_mir(global_isel=True)` uses `GlobalISelAbortMode::DisableWithDiag` (aborting mode would kill the process) and then **scans the result for residual `G_*` ops**, raising if selection didn't complete — because the fallback's diagnostic is only a warning. It also sets the GlobalISel flags per-call (AArch64 defaults `EnableGlobalISel=true`).
- These make the *IR→selected-MIR* path safe; they do **not** enable *hand-built-generic-MIR→selected-MIR*.

### What would unblock Route A (for discussion)

- A clean binding for `TargetPassConfig::setStartStopPasses` (or equivalent) to construct a "start at Legalizer, stop after emission" pipeline over externally-built generic MIR — replacing the process-global `start-*` `cl::opt` mechanism entirely.
- Running the GlobalISel legalize→select passes on hand-built MIR with the SelectionDAG fallback **disabled** and unselected ops surfaced as a catchable error (the residual-`G_*` scan from #588 is half of this already).
- Confirming the `MachineModuleInfoWrapperPass` build path (from `create_machine_function`) can carry generic MIR through those passes, and deciding what — if any — IR stub the functions need for the passes that still consult it.

Until one of those lands, **Route B is the supported way to reach native code from Python-built MIR**, and Route A generic-MIR lowering is only reachable indirectly by going back through IR (`run_codegen_to_mir`).

---

*Refs: #588 (GlobalISel gating + residual scan), #589 (build selected MIR), #590 (`emit_object` / `-start-after=finalize-isel`), and the deferred-items tracker #593 (item 1 covers the `start-after` `cl::opt` fragility).*

Contributor guide

No contributing guide indexed for this repository

Research direction

Read the Route A discussion alongside #588, #590, and #593, then inspect run_codegen_to_mir, create_machine_function, emit_object, and TargetPassConfig::setStartStopPasses. The work would need a defined pipeline for externally built generic MIR, disabled SelectionDAG fallback with catchable failures, and confirmation of the MachineModuleInfoWrapperPass and IR-stub behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.