Luminous-Dynamics / Luminous-Dynamics/symthaea
research-result: forbid positive confirmatory claims from being supported only by Missing/NotComputed outcomes
@Tristan-Stoltz-ERC is already working on this.
Since Sep 12, 2026.
- Dominant language
- Rust
- Stars
- 9
- Forks
- 1
- Avg merge
- 15m
- Merged PRs (30d)
- 8
Description
Context
Follow-up to PR #159 (symthaea-research-result) and generic research-integrity prerequisite for #1905 / #1898.
The current result layer correctly requires every preregistered primary metric to appear as an explicit outcome, including Missing { reason } and NotComputed { reason }. That is good.
However, the current claim validation checks only that a referenced metric id exists in the result manifest. It does not require the referenced metric to contain an observation capable of supporting the claim disposition.
Therefore a structurally valid manifest can currently express the equivalent of:
metric M = Missing("sensor failed")
claim C = ConsistentWithHypothesis
claim C references M
claim C interpretation = Confirmatory
The metric did not disappear, but the positive claim is still unsupported by the recorded observation state.
Core invariant
metric present
!=
metric observed
!=
metric supports positive/negative claim
A Missing or NotComputed primary result must never, by itself, support a confirmatory ConsistentWithHypothesis, InconsistentWithHypothesis, or NullResult claim.
Proposed evidence-state semantics
Classify MetricOutcome into an explicit support state, conceptually:
ObservedNumeric
ObservedBoolean
ObservedCategorical
Missing
NotComputed
and define conservative claim compatibility.
At minimum:
ConsistentWithHypothesisrequires at least one referenced observed metric or an explicitly typed observed/result artifact allowed by the frozen analysis plan;InconsistentWithHypothesislikewise requires observed evidence;NullResultrequires observed evidence sufficient to apply the preregistered null/no-effect decision rule; absence of measurement is not a null result;Inconclusive/NotEvaluatedmay be supported byMissing/NotComputedas appropriate;DescriptiveOnlyshould not be promoted into confirmatory hypothesis support merely by citing an analysis artifact.
Do not hard-code domain statistics into this generic crate. The generic layer should only enforce the distinction between observed evidence and absent/uncomputed evidence; domain/analysis code decides whether an observed value meets a preregistered criterion.
Primary metric completeness vs evidential adequacy
Preserve the current good rule:
all primary metrics must appear
but add a separate rule:
claim evidence must be semantically adequate for its disposition
These should remain distinct.
A campaign may therefore be procedurally confirmatory while yielding an inconclusive claim because a primary observation is missing. Confirmatory describes protocol lineage/analysis status; it must not imply that the scientific hypothesis received positive support.
Claim evidence projection
For each ResultClaim, derive a small evidence summary such as:
observed_metric_ids
missing_metric_ids
not_computed_metric_ids
observed_artifact_ids
Then validate disposition compatibility.
If artifact-only confirmatory hypothesis claims remain supported, require the artifact kind/role to be explicit and bound by the frozen analysis plan rather than treating any arbitrary digested artifact as evidentially interchangeable with an observed metric.
Adversarial regressions
At minimum:
ConsistentWithHypothesisciting onlyMissingmetric -> reject;ConsistentWithHypothesisciting onlyNotComputedmetric -> reject;InconsistentWithHypothesisciting only missing/uncomputed metric -> reject;NullResultciting only missing/uncomputed metric -> reject;Inconclusiveciting an explicit missing primary metric -> allow;NotEvaluatedciting explicit not-computed metric -> allow;- observed numeric metric may support a claim subject to existing protocol/reference checks;
- one observed + one missing metric remains explicit; missing evidence is not silently ignored in reporting;
- arbitrary
Analysisartifact alone cannot launder a missing primary metric into a positive confirmatory claim unless an explicit frozen artifact-evidence rule allows it; - imported/deserialized claim/outcome combinations receive the same semantic validation as constructor-created records.
Relationship to #1946
#1946 closes chronology/completeness/current-head integrity for amendments/deviations and recursively validates imported research evidence.
This issue is complementary: even a complete, correctly ordered result lineage can still make an evidentially invalid scientific claim if Missing is treated as though it were an observation.
Acceptance gate
- metric presence and metric observation are separate concepts;
- positive/negative/null hypothesis dispositions cannot be supported only by missing/uncomputed outcomes;
- missing/not-computed primary metrics remain first-class reported results;
- procedural confirmatory status does not imply positive hypothesis support;
- artifact-only hypothesis claims have an explicit conservative rule;
- claim compatibility is recursively validated on imported manifests;
- exact-head tests land in #1940 qualification scope.
Non-claims
This does not decide whether an observed effect is scientifically meaningful, statistically significant, causal, or generalizable. It prevents absence of observation from being represented as positive/negative/null evidence.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.