Validating and versioning community machine definitions
- Dominant language
- No language data
- Stars
- 5
- Forks
- 6
- Avg merge
- 4d 14h
- Merged PRs (30d)
- 2
Description
## Summary
`FreeCAD/Machines` distributes community `.fcm` machine definitions outside the main
source tree. This was done intentionally for several reasons:
1. Lower the bar for community contributed machines.
2. Reduce burden on maintainers
3. Make community created machines available between releases via the addon
These are real and worth keeping. Nothing below proposes moving the definitions back.
But there are some tradeoffs that are a problem
- **No CI.** Nothing checks a definition before or after merge.
- **No test harness.** There is no way to express "this machine produces this G-code" and
no way to notice when that stops being true.
- **Coupling to a repository we do not control.** The definitions are validated by CAM
core's loader, which changes on its own schedule. Breakage originates in one repository
and manifests in another.
- **Silent failure is the normal case.** A definition that loads but quietly drops half its
settings looks identical to a working one until somebody cuts a part.
- **Version skew.** Addon Manager delivers one branch to every FreeCAD version at once,
while the `.fcm` format keeps moving.
Already **one of the eight definitions currently in the repository does
not load at all**, and has not for some time. Nothing told us.
This issue proposes four things:
1. A validator that lives in CAM core, so it cannot drift from the loader it validates.
2. Golden G-code snapshots, which is the actual unit test we are missing.
3. A branch-per-FreeCAD-version distribution model, which the Addon Manager already
supports natively.
4. Treating `.fcm` as a published format, with the deprecation rules that implies.
---
## What is broken right now
I ran a prototype validator over every `.fcm` in the repository using the current
`Machine` model from CAM `main`:
| File | Finding |
| --- | --- |
| `Community/Generic/Generic_WireEDM.fcm` | **Fails to load.** `ValueError: 'wire_edm' is not a valid ToolheadType` |
| `Community/Dah Lih/MCV_510.fcm` | Invalid kinematics. `validate_kinematic_chain()` returns `['Duplicate sequence numbers in chain X']` |
| `Community/Generic/Generic_Smoothie.fcm` | Five keys the loader silently discards: top-level `blocks`, `postprocessor.args`, and `processing.{suppress_commands, spindle_wait, drill_cycles_to_translate}` |
The WireEDM failure is the one to design against. `ToolheadType` defines
`ROTARY`, `LASER`, `WATERJET`, `PLASMA` — there is no `WIRE_EDM`. A core enum member that
had already been serialized into a published data file was removed, and the community
machine referencing it has been dead ever since.
Note also what is **not** broken. Several files use older key names (`spindles` rather than
`toolheads`, `output.output_header` rather than `output.header.output_header`). Those load
correctly, because `Machine.from_dict` carries deliberate aliases for them. That fact
matters later — it is the mechanism the versioning proposal depends on.
---
## Three problems, not one
"We can't run tests" collapses three separate questions. They need different answers.
**1. Is the file well formed?** Valid JSON, required keys present, axis limits sane,
kinematic chain valid, referenced postprocessor exists, no duplicate machine names.
Static, cheap, catchable on every PR.
**2. Does it still do what the contributor tested?** A definition can load cleanly and
still emit different G-code than it did six months ago, because core changed. Only
comparing output catches this.
**3. Does it work on the FreeCAD versions we claim to support?** Independent of the other
two. A file can be perfect and still be unreadable by the version a user is running.
---
## Proposal 1: put the validator in CAM core
The instinct is to write a JSON-schema validator in the Machines repository. That creates
a second definition of what a valid `.fcm` is, which will drift from `Machine.from_dict`
— reproducing the exact problem we are trying to detect.
Instead, ship `Machine/models/validate.py` in CAM with a CLI entry point. Machines CI
installs FreeCAD and calls it. It stays in lockstep with the loader by construction, and
core gets to reuse it for the bundled `Machine/machines/` templates and for a **Validate**
action in the machine editor.
Report findings in three tiers:
- **error** — will not load, invalid kinematic chain, missing postprocessor, duplicate
display name, axis `min >= max`
- **warning** — keys present in the file that the loader ignores
- **info** — fields the model has added since the file was written
The warning tier deserves attention. It is the only check that catches silent behaviour
change, and it is what would have flagged Smoothie. The robust implementation is a small
core change: let `from_dict` optionally record which keys it consumed, then diff against
the input. That cannot go stale the way a hand-maintained key list would.
Two checks are correctness rather than style:
- **Machine names must be globally unique.** `MachineFactory.get_machine()` resolves by
display name and `list_configurations()` flattens namespaces. Two contributors both
shipping "Generic Mill" is a genuine collision.
- **`postprocessor_file_name` must resolve,** and every key in `postprocessor_properties`
must exist in that postprocessor's `get_full_property_schema()`.
A working prototype of these checks exists and already exits non-zero. It is what produced
the table above.
---
## Proposal 2: golden G-code snapshots
Validation proves a file parses. It does not prove the machine still emits the G-code the
contributor signed off on. For that, post a canonical path through each machine's declared
postprocessor and diff the result against a committed `.expected` file.
Design notes that keep this from becoming a burden:
- **Keep the test job in the harness, not per-machine.** One shared 3-axis path, one
4/5-axis path selected by `machine_type`. Contributors author no fixtures.
- **Normalize the header out.** Output contains timestamps and version strings. Post with
`--no-header` or filter.
- **Expect goldens to churn.** When core legitimately changes output, every golden moves.
That is the point: before a release ships, the diff shows maintainers exactly how every
community machine's output changed. But it needs a documented `--update-golden` step, or
reviewers will route around it.
---
## Proposal 3: distributing across multiple FreeCAD versions
This is the part with a concrete mechanism already available.
### The constraint
Backward compatibility in `from_dict` only helps in one direction: **new FreeCAD reading
old files**. It does nothing for **old FreeCAD reading new files**, and no amount of core
work fixes that retroactively for versions already released.
So a single branch served to every FreeCAD version can only ever target the oldest format
that all supported versions can read.
### What Addon Manager already supports
`AddonCatalogEntry` (FreeCAD 1.1+) holds `freecad_min`, `freecad_max`, and `git_ref` per
entry, and an addon may have **several** entries. `is_compatible()` selects the entry
matching the running FreeCAD version and installs from that branch.
Branch-per-FreeCAD-version is therefore a supported distribution model, not a workaround.
### Proposed model
- `main` is the development branch, targeting current FreeCAD dev.
- Cut a release branch per supported FreeCAD line: `FreeCAD-1.2`, `FreeCAD-1.3`, and so on.
- Register one catalog entry per branch with the matching `freecad_min`/`freecad_max`.
- CI validates each branch against the FreeCAD versions that branch actually claims.
- **Contributors only ever open PRs against `main`.** Maintainers backport by cherry-pick,
and CI proves the backport loads and posts correctly on the older version.
Because the loader is backward compatible, most definitions are byte-identical across
branches and backporting is a plain cherry-pick. Branches only diverge when a genuinely
breaking format change lands — which should be rare, and which the next section is about.
### The cheaper half of the same idea
For most machines, a file written in the oldest supported format works on every version,
since newer loaders read older keys. Where that holds, prefer it: it keeps branches
identical and backporting trivial. The branch mechanism is there for when it does not hold,
not as the everyday path.
---
## Proposal 4: treat `.fcm` as a published format
CI would have told us about WireEDM sooner. A policy would have prevented it.
An enum serialized into a distributed data file cannot be removed silently. `ToolheadType`
needs a deprecation path: keep the member, mark it deprecated, migrate on load. More
generally:
- Changes to `.fcm` should be additive. Renames ship with an alias, as `spindles` and
`output_header` already do.
- The `version` field already present in every file should actually gate something.
- Removing a serialized enum member or a read key is a breaking change and needs a
migration, a release note, and a Machines-repo backport.
### Closing the loop the other way
Nothing inside the Machines repository can catch a core change before it merges. Two steps:
- **Now:** a nightly run of Machines CI against FreeCAD `main`, opening or updating an
issue on failure. Catches breakage within a day.
- **Later:** a job in FreeCAD's own CI that checks out `FreeCAD/Machines` and validates it,
so breakage surfaces in the PR that causes it. Run it non-blocking at first — a community
data repository should not gate core PRs until the validator has earned trust.
---
## Smaller items
- Warn when a file's `freecad_version` predates the oldest supported release, prompting a
re-save through the current model. This flushes drift out of files one at a time.
- `CODEOWNERS` per vendor directory, which directly serves the goal of distributing review.
- Validate that a file's path agrees with its `manufacturer` and `name` fields.
- A PR template asking which FreeCAD version was used, which physical machine this
describes, and whether it has been cut on hardware.
---
## What has to change, by repository
Nothing here is sequenced as a single project. The core items unblock most of the Machines
items, and the existing-machine fixes are independent of both.
### FreeCAD core (`FreeCAD/FreeCAD`)
**Required for any of this to work**
- **Add `Machine/models/validate.py` with a CLI entry point.** The checks are the three
tiers described in Proposal 1. A working prototype exists and can be adapted.
- **Let `from_dict` report which keys it consumed.** Small, optional-argument change.
Without it the "keys the loader ignores" warning has to be a hand-maintained list, which
will go stale.
**Format correctness — these are bugs today**
- **Restore `wire_edm` to `ToolheadType`**, marked deprecated, or provide a documented
migration. A published file references it. Until this lands, `Generic_WireEDM.fcm` cannot
be fixed in the Machines repo alone.
- **Fix `machine.py:1555`.** `config.postprocessor_args = post_data.get("args", "")`
assigns to an attribute that is not a `Machine` field, so `to_dict()` never writes it and
nothing reads it. Either wire it through or drop the read and let the validator warn.
- **Decide what the `version` field means** and gate on it. It is written into every file
and currently checked by nothing.
**Supporting work**
- **A `Validate` action in the machine editor**, running the same validator, so contributors
catch problems before opening a PR.
- **`Machine/machines/` does not exist in the source tree**, though `CMakeLists.txt`
references it for both copy and install. Bundled templates would be validated by the same
tool.
- **`Machine.add_spindle()` (`machine.py:841`) passes positional arguments into
`Toolhead(name, toolhead_type, id, ...)` in the wrong order**, so `id` lands in
`toolhead_type`. Any harness that builds machines programmatically hits this.
- **Later:** a non-blocking core CI job that validates `FreeCAD/Machines`.
### Machines repo (`FreeCAD/Machines`)
**CI**
- GitHub Actions workflow installing FreeCAD across a version matrix and running the core
validator. Changed files on PRs, full tree on a schedule.
- Scheduled run against FreeCAD `main` that opens or updates an issue on failure.
- Practical note for whoever writes this: `FreeCADCmd script.py ` tries to *open*
extra arguments as documents. Pass configuration through the environment.
**Test harness**
- Shared canonical job — one 3-axis path, one 4/5-axis path selected by `machine_type`.
- One `.expected` golden per machine, plus a documented `--update-golden` bless step.
**Branch and release structure**
- Cut `FreeCAD-1.2`, `FreeCAD-1.3` (or whichever lines we commit to supporting) from `main`.
- Per-branch `package.xml` with the matching ``/``.
- Document that contributors target `main` only, and that maintainers backport.
**Governance and docs**
- `CODEOWNERS` per vendor directory.
- PR template: FreeCAD version used, which physical machine, whether it has been cut on
hardware.
- `Documentation/Definition.md` and `.github/CONTRIBUTING.md` are both stubs. They need the
authoring workflow, how to validate locally, and what CI enforces.
- Decide whether `Community/testmachines/` belongs in the distributed tree or in a fixtures
directory excluded from the catalog.
### Addon catalog (`FreeCAD/Addons`)
Easy to miss, since it is a third repository. The catalog is `Data/Index.json`.
- Add one entry per Machines branch, each with `freecad_min`, `freecad_max`, and `git_ref`.
`AddonCatalogEntry.is_compatible()` does the selection at install time.
### Existing machine definitions
| File | Action | Blocked on |
| --- | --- | --- |
| `Generic/Generic_WireEDM.fcm` | Restore loading — either core re-adds `wire_edm`, or the file moves to a supported toolhead type | Core decision |
| `Dah Lih/MCV_510.fcm` | Fix duplicate sequence numbers in the X chain | Someone who knows the machine |
| `Generic/Generic_Smoothie.fcm` | Remove or migrate the five ignored keys; decide whether `postprocessor.args` should become `postprocessor_properties` | Core fix to `:1555` for the `args` half |
| All machines | Add a golden `.expected` once the harness exists | Harness |
**Do not bulk re-save every file through the current model.** It is the obvious cleanup and
it is wrong: re-saving through current dev writes the newest format, which older FreeCAD
may not read, which is precisely the compatibility direction we cannot fix. The older key
names in these files load correctly today via the aliases in `from_dict`. Normalize a file
only when the validator flags a real problem in it.
---
## Questions for discussion
1. Does the validator belong in CAM core, or is the coupling to a specific FreeCAD version
worse than the drift risk of a standalone one?
2. Which FreeCAD versions do we commit to supporting? That number sets the branch count and
the CI matrix size.
3. Are golden G-code snapshots worth the churn, or is load-and-validate enough for now?
4. Who blesses golden updates when a core change moves every machine's output?
5. Should FreeCAD core CI validate `FreeCAD/Machines`, accepting a cross-repository
dependency in core's pipeline?
6. What do we do with the three broken definitions today — fix, or remove pending a
maintainer?
Contributor guide
Research direction
Start by reading the proposed FreeCAD core entry points: Machine/models/validate.py, Machine.from_dict, machine.py:1555, and Machine.add_spindle() at line 841. Compare those changes with the Machines repository's CI and Addon Manager branch model. Done requires agreed validation checks, version compatibility, golden-output coverage, and a rollout plan across both repositories.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, release, testing, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100