control-toolbox / control-toolbox/CTParser.jl

Align CTParser with CTBase 0.29 / CTModels 0.18 / ExaModels 0.12

Open
#325 2 comments 0 reactions 1 assignee Claimed by @jbcaillau View on GitHub
Dominant language
Julia
Stars
3
Forks
0
Avg merge
4h 22m
Merged PRs (30d)
2

Description

## Why now

CTParser's compat bounds are two ecosystem generations behind:

| | CTParser declared | released |
| --- | --- | --- |
| CTBase | `0.18, 0.27, 0.28` | 0.29.3 |
| CTModels | `0.10, 0.14, 0.15` | 0.18.0 |
| ExaModels | `0.9` | 0.12.0 |

CTSolvers 0.5.3 and CTFlows 0.17.2 are already out and aligned on CTBase 0.29 / CTModels 0.18.

## Why CTParser is the blocker

CTSolvers 0.5.3 already declares `ExaModels = "0.12"`, but its `CTSolversExaModels` extension only reads backend metadata (base type, execution backend). The package that actually **emits** `ExaCore` / `variable` / `constraint` / `objective` calls is CTParser, from `def_exa` in `src/onepass.jl`.

ExaModels 0.12 **deleted** that mutable builder API — `src/deprecated.jl` is gone, and with it `variable`, `parameter`, `subexpr`, `constraint!` and `LegacyExaCore`. So CTSolvers' declared 0.12 support was nominal until this landed: any `:exa` solve failed at run time, inside generated code, with `UndefVarError: variable not defined in ExaModels`.

CTParser is the one package in the stack whose **source** had to change for the ecosystem to reach ExaModels 0.12.

## Evidence

Measured against ExaModels 0.12.0 + MadNLP 0.10.1 with a throwaway capability probe, plus a `:fun` smoke build against CTBase 0.29.3 + CTModels 0.18.0 — not inferred from release notes.

**The whole target set co-resolves**: CTBase 0.29.3, CTModels 0.18.0, ExaModels 0.12.0, CUDA 6.3.0, MadNLP 0.10.1, MadNLPGPU 0.10.2, OrderedCollections 2.0.1, NLPModels 0.21.12, Interpolations 0.16.3, KernelAbstractions 0.9.42. The remaining `--outdated` entries (`Parsers` held by JSON, `TimerOutputs` held by LinearOperators) are transitive and not ours to fix.

**`src/` needed no change for CTBase 0.29 / CTModels 0.18 / OrderedCollections 2.** CTBase 0.29 dropped its top-level exports, but its submodules are `using`-ed inside `CTBase`, so the two symbols `src/` uses — `CTBase.ctindices` and `CTBase.ctupperscripts` — still resolve, as does `ParsingError`, which generated code reaches through `e_prefix`. Confirmed in the real suite: the `:fun` groups stayed green at 1054/1054 across the bump.

**The functional emission reproduces the old one exactly.** Replacing `variable`/`constraint`/`objective` with `add_var`/`add_con`/`add_obj` — each returning `(new_core, result)`, so the generated code threads and rebinds the core — gives the same model (`nvar=303`, `ncon=204`) and the same optimum (6.0006 against the analytic 6) on `:euler`, `:euler_implicit` and `:midpoint` for a reference double integrator. Every awkward CTParser-specific shape survives the threading: rebinding inside `__wrap`'s `try/catch`, rebinding inside a `for` body, `core, dyn_con[i] = if scheme == … end`, `Vector{ExaModels.Constraint}(undef, n)`, `add_obj(core, node)` with no generator, and `add_con(core, node; lcon, ucon)`.

**Rebinding the core is free.** With the 0.12 default (`concrete = nothing`, `Vector{Any}` block storage) `typeof(core)` is *invariant* across every `add_*`. With `concrete = Val(true)` the core's type changes on each `add_*`, which is why that keyword is the wrong choice here — see the note on #323 below.

## Public API changes

### 1. The `:exa` emission requires ExaModels ≥ 0.12

Supporting both APIs is not practical: CTParser does not depend on ExaModels (the module is reached through `prefix_exa()`), so it cannot branch on the version at macro-expansion time.

```julia
# Before — emitted against ExaModels 0.9–0.11
c = ExaModels.ExaCore(base_type; backend, minimize)
x = ExaModels.variable(c, n, 0:grid_size; lvar, uvar, start)
ExaModels.constraint(c, expr for j in 0:grid_size-1; lcon, ucon)
ExaModels.objective(c, expr for j in 0:grid_size-1)

# After — emitted against ExaModels 0.12
c = ExaModels.ExaCore(base_type; backend, minimize) # unchanged
c, x = ExaModels.add_var(c, n, 0:grid_size; lvar, uvar, start)
c, _ = ExaModels.add_con(c, expr for j in 0:grid_size-1; lcon, ucon)
c, _ = ExaModels.add_obj(c, expr for j in 0:grid_size-1)
```

### 2. Untyped `String` errors become `CTException` subtypes

```julia
# Before
catch e
e == "unknown numerical scheme: gauss_legendre_2 (possible choices are ...)"
end

# After
catch e
e isa CTBase.IncorrectArgument
end
```

## Plan

Released as **0.9.0**, joining the rest of the ecosystem out of beta. Landing as one draft PR (#326), one commit per phase.

- [x] **A** — compat bounds in `Project.toml`, `test/Project.toml`, `docs/Project.toml`; no source change.
- [x] **B** — migrate the 23 ExaModels emission sites in `src/onepass.jl` to the functional builder API, plus the one direct `ExaCore` use in `test/test_exa_linalg.jl`.
- [x] **B2 (unplanned)** — carry ExaModels' unregistered linear-algebra extension; see the comment below.
- [x] **C** — typed exceptions across the whole `throw(String)` class (17 sites), and the 19 `@test_throws String` assertions that locked in the old behaviour.
- [x] **D** — qualify the ExaModels import in `test/runtests.jl`.
- [x] **E** — full suite 2580/2580, docs build clean.
- [x] **F** — `CHANGELOG.md` + `BREAKING.md` (retroactive bootstrap from the last non-beta tag, v0.8.15), version bump to 0.9.0.
- [x] **CI** — the retired `kkt` self-hosted runner replaced by `occidata`, trigger labels renamed to the ecosystem's `run ci ` form.

## Closes

- Closes #323 — but note the fix is **the upgrade itself**, not the `concrete = Val(true)` suggested in that issue. Measured under 0.12: `ExaCore(Float64; backend, minimize)` emits no warning at all, because the deprecation shim it came from no longer exists. `concrete = Val(true)` would silence nothing extra *and* would reintroduce per-block recompilation. Once this is released, the `with_logger(NullLogger())` workaround can come out of the OptimalControl documentation ([control-toolbox/OptimalControl.jl#877](https://github.com/control-toolbox/OptimalControl.jl/issues/877)).
- Closes #322 — widened from the three scheme sites cited there to the whole defect class, so no `throw(String)` is left in `src/`.
- Closes #230 — `test/runtests.jl` did `using ExaModels` while `constraint` was imported from CTModels. Still true under 0.12, where `constraint` remains exported for the oracle form.

## Follow-ups

- **CTDirect.jl still pins `ExaModels = "0.11"`** and consumes this generated code — it needs a matching bump once this is released.
- **Full "`using`, never `import`" sweep** (Handbook tenet 2). `src/CTParser.jl` has 5 bare `using` and `test/runtests.jl` uses `import`. Kept out of this PR deliberately: the sweep is ~36 call sites in `src/` but ~523 in `test/`, and 26 of the `src/` ones are `@match` in `src/onepass.jl` — the very file phase B restructures. Two unrelated rewrites in one diff is unreviewable, and the ecosystem's own precedent is a dedicated release (CTBase 0.29.1, 174 files, "No public API, type, or signature changes").
- **ExaModels' unregistered extension** — [madsuite-org/ExaModels.jl#323](https://github.com/madsuite-org/ExaModels.jl/issues/323). Our port in `ext/CTParserExaModels.jl` is temporary and its lifetime depends on that answer.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.