-j runs the whole frontend twice for a program outside the flat subset
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 14
- Forks
- 2
- Avg merge
- 12h 42m
- Merged PRs (30d)
- 61
Description
vxc -j N runs the parallel frontend, and when the flat emitter will not take the program it
hands back Ok(None). The driver answers that by calling execute_vx_pipeline, which loads the
same files from disk again and redoes the whole frontend single-threaded.
Nothing of the first frontend survives. It has already loaded and parsed every module,
macro-expanded, resolved names, frozen the registry, built the env and type-checked every
function by the time codegen declines, and all of it is dropped:
// src/driver.rs, execute_one
if self.execute_parallel(main_file, &filename, &mlir_args, jobs)?.is_some() {
Self::report_phases();
return Ok(());
}
let emitted = self.execute_vx_pipeline(main_file, &filename, &mlir_args);
So for a program outside the flat subset, -j is strictly more work than no flag. On an
8,000-line program whose only declining construct is a generic method returning a generic enum
out of a match:
$ vxc --emit-mlir big_decliner.vx # median of 15, release build
0.280s
$ vxc -j 4 --emit-mlir big_decliner.vx
0.300s
The phase table says where the extra 20 ms goes. These are the parallel frontend's own phases on
that run, and every one of them is work the sequential compile then repeats:
$ VX_PIPELINE_PHASES=1 vxc -j 4 --emit-mlir big_decliner.vx
phase,parse,4.0435
phase,macro_expand,1.7306
phase,type_check,3.0377
phase,codegen,1.1798
phase,teardown,0.6251
The same program with the declining construct removed is the control, and there -j is worth
having: 0.10s without the flag against 0.09s with it.
Two things follow. The cost is proportional to program size, because it is the entire frontend
rather than a fixed overhead, so it grows with the corpus rather than staying at 20 ms. And
nothing tells the user it happened -- the decline line says the sequential driver is taking
over, not that the work so far was discarded.
Fixing it properly means the AST codegen consuming what the parallel frontend already produced
-- its modules, its check results, its frozen registry -- rather than the sequential driver
starting from the file paths. That is an architectural change to how the two codegen paths get
their input, not a local fix, which is why it is filed rather than folded into #625.
A smaller step that is not a fix but removes the surprise: say on the decline line that the
frontend's work is being redone, so a -j build that got slower explains itself.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/driver.rs at execute_one, especially the transition from execute_parallel to execute_vx_pipeline. Trace what the parallel frontend produces and what the AST codegen path consumes, using VX_PIPELINE_PHASES=1 to observe repeated work. Done means the sequential path no longer reloads and reruns the frontend for programs outside the flat subset, or the decline message clearly reports that redo if only the smaller step is chosen.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- compilers
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100