Open-MBEE / Open-MBEE/sysmlv2-testing
Independent replication wanted: Pilot Implementation on another machine
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- Avg merge
- 29m
- Merged PRs (30d)
- 1
Description
Every pilot-implementation run in this ledger was executed on one machine (macOS 15, arm64). Until someone else runs it, the ledger cannot separate "this is how the Pilot behaves" from "this is how it behaves on that laptop".
This is as much an adapter shakedown as a reproduction. The Pilot adapter is the most bespoke of the three — a small Java shim over SysMLInteractive, driven headlessly — and it has the heaviest setup. If it doesn't work on your machine, that is the finding: please open an issue rather than working around it.
Fair warning that this is the highest-friction of the three implementations. If you want the easy one first, see #1.
What to do
git clone https://github.com/Open-MBEE/sysmlv2-testing.git
cd sysmlv2-testing && uv sync
# JDK 21 SPECIFICALLY -- see the script header for why 26 does not work
export JAVA_HOME=/path/to/jdk-21
PILOT_REPO=/path/to/SysML-v2-Pilot-Implementation \
PILOT_COMMIT=692170b71867353b8f90341e61556f49a5beb0e5 \
toolchain/get-pilot-jar.sh # Maven/Tycho reactor build; not quick
export PILOT_GLUE_CLASSPATH=... # the script prints this line
export SYSML_LIBRARY_DIR=/path/to/SysML-v2-Pilot-Implementation/sysml.library
uv run svt party add --id <your-machine-id> --label "<OS, arch>"
You will be refused on the first run, and that is expected here in a way it isn't elsewhere. The Pilot ships no release artifact — everyone builds their own jar, and Maven builds are not byte-reproducible, so every builder's digest differs. svt run prints yours and the command to add it:
uv run svt version add-artifact-digest --implementation pilot-implementation \
--version 692170b71867353b8f90341e61556f49a5beb0e5 --digest <yours>
Worth being honest that this makes the digest pin weak for the Pilot specifically: the set grows by one per builder and cannot distinguish "the same build" from "a build of the same commit". It still tells you two runs used the same jar, which is not nothing. If you have a better idea, that's a welcome issue in itself.
Then:
uv run svt run --testcase end-feature-redefinition-explicit \
--implementation pilot-implementation \
--version 692170b71867353b8f90341e61556f49a5beb0e5 \
--as <your-machine-id>
Open a PR with the resulting ledger diff. Please don't hand-edit any .ttl — svt is the only writer.
Why this test case
end-feature-redefinition-explicit is where the Pilot is the odd one out. On connection l : Link { end :>> source = a; end :>> target = b; } — the spec's own normative example form — OpenSysML and sysml-toolkit both accept it; the Pilot rejects it with "Must have at least two related elements", so it records failed.
We already checked that isn't a harness artifact: the fixture was fed both as separate indexed resources and as one concatenated compilation unit, with the same verdict either way. But that was checked on one machine, by the person who wrote the adapter. An independent run is the thing that would actually settle it.
What should happen
Agreement writes a svt:Reproduction — not a second TestRun. Disagreement is recorded as its own TestRun with a warning naming what it contradicts, and nothing is retracted. A disagreement here would be a genuinely interesting result.
Known rough edges, so you can tell a bug from a papercut
- JDK 21, not "whatever you have." JDK 26 makes
org.omg.sysml's Xtend compilation fail with ~150,000 errors. This is understood, not flaky; retrying will not help. - Without
SYSML_LIBRARY_DIRthe Pilot loads no standard library and still returns verdicts — confidently wrong ones. The adapter refuses in that state now, but unresolvedScalarValues/Parts::Partin captured output is the tell. - The fat jar is ~137 MB; the first Maven run downloads a lot of Eclipse p2 metadata.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with toolchain/get-pilot-jar.sh and its header, then follow the documented JDK 21 setup and build the pinned Pilot commit. Run the svt commands for the end-feature-redefinition-explicit testcase on a separate machine, recording any digest and machine details as instructed. Done means opening a PR containing the resulting ledger diff without manually editing .ttl files.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, python, shell
- Domain
- testing-qa, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 58/100