Open-MBEE / Open-MBEE/sysmlv2-testing
Independent replication wanted: sysml-toolkit v0.6.0 on another machine
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- Avg merge
- 29m
- Merged PRs (30d)
- 1
Description
Every sysml-toolkit run in this ledger was executed on one machine (macOS 15, arm64). That is a real limitation: the ledger currently cannot separate "this is how sysml-toolkit behaves" from "this is how it behaves on that laptop". An independent run is the only thing that settles it.
This is as much an adapter shakedown as a reproduction. If it doesn't work on your machine, that is the finding — please open an issue rather than working around it. The two bugs found this way so far were both real and both invisible from one machine.
What to do
git clone https://github.com/Open-MBEE/sysmlv2-testing.git
cd sysmlv2-testing
uv sync
# the pinned release binary for YOUR platform (prints the sha256 to pin)
TAG=v0.6.0 PLATFORM=<your-platform> toolchain/get-sysml-toolkit.sh
export SYSMLV2_BIN=... # the script prints this line
# the release asset ships no standard library -- verified, not assumed.
# Point this at a SysML-v2-Release checkout, or the Pilot's sysml.library/
export SYSMLV2_LIB_DIR=/path/to/sysml.library
uv run svt party add --id <your-machine-id> --label "<OS, arch>"
You will be refused on the first run, and that is correct. The release ships one artifact per platform, so your binary's sha256 differs from the one pinned here (32dcc653…, aarch64-apple-darwin) even though it is the same release. svt run prints both hashes and the command to add yours:
uv run svt version add-artifact-digest --implementation sysml-toolkit \
--version 3a13c64adb93f1d069ce021c598318587126044a --digest <yours>
Then:
uv run svt run --testcase redefinition-ambiguity-2-resolution \
--implementation sysml-toolkit \
--version 3a13c64adb93f1d069ce021c598318587126044a \
--as <your-machine-id>
Open a PR with the resulting ledger diff. Please don't hand-edit any .ttl — svt is the only writer, and a hand-authored record is exactly what this ledger exists to avoid.
Why this test case
redefinition-ambiguity-2-resolution records failed here: the two anonymous :>> items redefinitions resolve to each other rather than to the inherited base feature (Open-MBEE/sysml-toolkit#2). Confirming a failure independently is worth more than confirming a pass — if it does not reproduce on your machine, then either the bug is environment-specific or our adapter is wrong, and both of those are things we want to know.
If you have appetite for more, the other seven are listed in svt view --implementation sysml-toolkit.
If a newer release lands, run that instead — and I'll reproduce yours
If sysml-toolkit cuts a new release and you'd rather exercise that than
v0.6.0, please do; it's the more useful thing. Register it and run against
it:
uv run svt version add --implementation sysml-toolkit --commit <sha> --label <tag>
uv run svt version add-artifact-digest --implementation sysml-toolkit \
--version <sha> --digest <your platform's sha256>
uv run svt run --testcase <id> --implementation sysml-toolkit \
--version <sha> --as <your-machine-id>
Then I'll pull the same release, add the macOS/arm64 digest alongside
yours, and reproduce your run from here — which is the reciprocal half of
this request and the direction that actually matters: a new release
confirmed by two parties on two platforms from the day it ships, rather
than one machine's word for it months later.
Worth being precise about what that produces, since it is not the same as
the v0.6.0 case above. A new release is a new svt:Version, so your
run against it is a new TestRun — not a reproduction of anything, because
nothing has tested that version before. My run of the same version is
then the svt:Reproduction of yours. The ledger keeps the two roles
distinct on purpose: a first observation and an independent confirmation
are different kinds of evidence.
What should happen
Agreement writes a svt:Reproduction — not a second TestRun. Re-running a test confirms a claim; it does not produce new evidence about the tool, and recording it as a new run would be duplication. Disagreement is recorded as its own TestRun with a warning, and nothing is retracted.
Known rough edges, so you can tell a bug from a papercut
- The v0.6.0 release asset genuinely does not vendor a standard library. Without
SYSMLV2_LIB_DIRthe tool resolves nothing and still returns a verdict — a confident wrong answer, not an error. The adapter refuses in that state now, but if you see unresolvedScalarValues/Parts::Partin captured output, that's the cause. - Two versions are registered: the v0.6.0 release tag, and a local build from
main@29d57f4whose tree is identical. They are separate Versions because they are different binaries. Use the release one above.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Clone the repository, run uv sync, and use toolchain/get-sysml-toolkit.sh for your platform. Register your machine and platform digest, then run testcase redefinition-ambiguity-2-resolution with uv run svt run; inspect the resulting ledger diff and open a PR without hand-editing .ttl files.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, shell
- Domain
- cli, testing-qa
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100