arkavo-org / arkavo-org/VRMMetalKit

vrm-author-cli corpus evidence: follow-ups, and whether the authoring tool should split out

Open
#440 0 comments 0 reactions 0 assignees View on GitHub
documentation technical-debt
Dominant language
Swift
Stars
6
Forks
2
Avg merge
18h 51m
Merged PRs (30d)
26

Description

Follow-up from the corpus evidence work on `vrm-author-cli` (pushed at `f844973`, 34 commits).

That branch added a corpus evidence path for acceptance packs whose runner is an XCTest suite, and generalised it so a second aesthetic can be added as data rather than code. The headline outcome is a truthful negative: the witness generator ran to completion over 18 body families and **zero are reachable** by the `native-anime-v1` template, so `build`, `control set`, `export vrm` and `recipe apply` each declare the corpus dimension required with a floor of 1 and report it as fail.

Current state: 133 runner tests, 33 linter, 25 generator, 303 Swift. The 35-pack sweep is 28 pass, 3 pending, 4 fail.

## Do not squash-merge this branch

All 35 packs pin commit `b9cc9ac` at `runner.environment.pinnedCommit`, and that commit exists only in this branch's history. A squash or rebase onto `main` makes every pin unresolvable, and `test_every_shipped_pack_pins_a_commit_carrying_its_oracles` will fail.

Merge with history preserved, or re-pin immediately after with:

```
python3 scripts/repin_packs.py # repairs broken pins
python3 scripts/repin_packs.py --check # reports without writing
```

## Decide what to do about the failing corpus dimension

Four commands now declare corpus evidence they do not have. The binding constraint is eye spacing relative to head width: the template starts near 0.265 of head width and its controls bottom out near 0.208, while 16 of the 18 families sit between 0.130 and 0.166. Of the remaining two, `alicia-solid` sits at 0.2641 and is blocked instead by shoulder width with its control at the rail, and `heroes-alex` is rejected before the CLI because its height of 1.1764 m is outside the template's valid range of [1.2, 2].

Either change the template's eye spacing relative to head width, which is the only lever that makes families reachable, or accept the failure as the standing record. No pack-layer change can move it. Regenerating witnesses after a template change takes over an hour and rebases four pack hashes.

## The replay path is stub-tested only

With zero families eligible, nothing downstream of the eligibility check executes against real data: the replay, the per-operation assertions and the witness provenance gate are all proven against a stubbed CLI and a stubbed linter. This resolves itself the moment one family becomes reachable, and not before. Noted in spec §4.2.

## Port the style linter's measurement into Swift

`StyleToolchain` spawns `python3 scripts/style_lint.py` at runtime, so the shipped `style lint` command has a hard dependency on a Python interpreter and on a script living outside the package. This is the root of most of the pinning friction: a comment-only edit to that script replaces the four packs' truthful negative with a provenance mismatch until a re-solve of over an hour.

Porting the measurement to Swift would remove the interpreter dependency from a shipped command, collapse the hash cascade, and let the generator and the grader share one implementation. It requires reconciling the two implementations against the pinned 22-asset corpus before anything can be trusted.

Note the scope: only `style_lint.py` ships. `corpus_witness.py`, `acceptance_run.py`, `repin_packs.py` and their tests are build-time evidence tooling and are fine as Python.

## Should the authoring tool move to its own repository?

Probably yes, and the seam is clean. `VRMAuthorKit` has no dependencies at all; only `VRMAuthorRender` and `VRMAuthorKitTests` touch `VRMMetalKit`. A split makes the tool depend on the library, which is the right direction and an ordinary package dependency.

The case: a Python runtime dependency is unremarkable in an authoring tool's repository and a wart in a Swift rendering package. Today anyone depending on the renderer also pulls the authoring targets, 35 acceptance packs, a style corpus and a pinning harness, none of which touch Metal.

The costs: every pack pins Swift test files by path, so a move forces a full re-pin, which the script now handles. More seriously, the provenance chain does not survive a fresh repository. Pinned commits reset, and keeping them checkable means filtering history rather than copying files. Better done before more provenance accumulates.

Counter-argument: if the project is still winding down, porting the linter buys more than splitting repositories.

## Smaller open items

- The visual dimension has no judge, so three packs stay pending indefinitely. Predates this work and is why nothing reaches the top evidence level.
- `run_cli` in `corpus_witness.py` raises `FileNotFoundError` on a launch failure. Classification is correct; only the docstring named the wrong type, now fixed.
- `reach_holds` in the runner is stricter than the generator's own eligibility test, because the generator skips unscorable metrics. Verified inert today, since all 14 target metrics carry two-sided range rules. Revisit when `corpus_witness.py` is next touched, so an eligible witness can never fail replay on a metric the solver never scored.
- Re-pinning is now scripted but still manual. A pre-commit or CI check would close the loop that let 31 of 115 oracle entries drift unnoticed.

Contributor guide

Open the contributing guide

Research direction

Start by reading the `vrm-author-cli` acceptance runner, `scripts/style_lint.py`, `scripts/repin_packs.py`, and spec §4.2; run the stated pack and runner tests to reproduce the current evidence. The issue contains several independent design choices, so done requires an agreed scope, preserved or repaired pack provenance, and passing evidence checks after the selected change.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, swift
Domain
build-system, testing, tooling
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.