cqframework / cqframework/cql-tests-runner

Remove the stale `RolledOutIntervals` skip reason from the shipped configs

Open
#133 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
5
Forks
3
Avg merge
2d 16h
Merged PRs (30d)
2

Description

Remove the stale RolledOutIntervals skip reason from the shipped configs

Repository: cqframework/cql-tests-runner
Blocked on: cqframework/cql-tests#146 (approved, not yet merged) and a submodule bump.
Related: #80 — see "The fifth config is different" below.

Summary

Four of the shipped configurations skip CqlAggregateTest:AggregateTests:RolledOutIntervals with
this reason:

CQLtoELM - Could not resolve identifier MedicationRequestIntervals in the current library

That identifier has not appeared in the test since 2024-08. The expression it referred to was
replaced with an inline list of intervals, so the reason describes a translation failure that can no
longer occur. The skip has outlived its cause.

Affected files, all carrying the identical obsolete reason:

  • conf/localhost.json
  • conf/development.json
  • conf/smile-cdr-local.json
  • conf/localhost-evaluate.json

Because these ship with the runner, the obsolete message is reproduced in every recorded run in
cql-tests-results — cqf-java, cqf-javascript, firely and cozeva all report:

"testStatus": "skip",
"skipMessage": "Skipped by config: CQLtoELM - Could not resolve identifier MedicationRequestIntervals in the current library"

It is also carried into local reporting, e.g. docs/test-run-evaluation-2026-08-19.md lists it as
one of the 14 skips.

The fifth config is different

conf/cql-execution-local.json skips the same test, but for a current and correctly attributed
reason:

"reason": "Test Runner Not Detecting Equal Lists of Interval<DateTime>/Period: https://github.com/cqframework/cql-tests-runner/issues/80"

That is not stale in the same sense. It should still be re-evaluated, because #129 has since
merged support for unwrapping a List<...> wrapper on cqf-cqlType, which is the capability #80
was asking for. If that resolves #80 for this test, this entry can go too — but on its own evidence,
not as part of the cleanup below.

Sequencing: do not simply delete the entries yet

cql-tests#146 changes the aggregate's starting type from List<Interval<DateTime>> to
List<Interval<Date>>, so the result is a list of Date intervals matching the declared output. That
is a prerequisite for this test producing the expected value at all.

However, running the expression against a reference engine (HAPI FHIR 8.10.0 / CQF engine 4.1.0,
$cql, CQL 1.5) fails to translate with both the current and the #146 type:

Expected an expression of type 'decimal', but found an expression of type 'System.Integer'.,
Could not determine signature for invocation of operator System.Add.,
Internal translator error., Internal translator error.,
Could not determine signature for invocation of operator System.Union.

The Decimal/Integer complaint appears to come from
Quantity{ value: duration in days of X, unit: 'days' }duration in days of X yields an
Integer where Quantity.value expects a Decimal. That is independent of both the stale reason and
of #146.

So removing the skip today would convert a skip into a failure, not a pass, on at least this
engine. That may still be the right outcome — an honest failure is more useful than a skip citing a
cause that has not existed for two years — but it should be a deliberate choice rather than a
surprise in the next results run.

Proposed steps

  1. Wait for cql-tests#146 to merge, then bump the cql-tests submodule (currently pinned at
    727219f).
  2. Re-run RolledOutIntervals unskipped against the reference engine.
  3. If it passes, delete the four stale entries.
  4. If it still fails on the Decimal/Integer translation error, either raise that upstream against
    the test or the engine, and replace the four reasons with one that names the actual current
    cause and links the new issue.
  5. Separately, re-evaluate conf/cql-execution-local.json's #80-based skip now that #129 has
    merged.

Either way the outcome is the same for this issue: no shipped config should carry a skip reason
that no longer describes reality.

Verification notes

  • The obsolete identifier does not appear anywhere in the current test:
    grep -rn "MedicationRequestIntervals" cql-tests/tests/cql/ returns nothing.
  • conf/smile-cdr-local.json additionally contains a JSON syntax error (a trailing comma after
    "cqlEngineVersion"), so that file currently fails to parse and cannot be loaded at all. Worth
    fixing while these entries are being edited, though it is a separate defect.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Wait for cql-tests#146, bump the cql-tests submodule from 727219f, and inspect the four affected files: conf/localhost.json, conf/development.json, conf/smile-cdr-local.json, and conf/localhost-evaluate.json. Re-run RolledOutIntervals unskipped against the reference engine. Done means no shipped config has a stale reason; if the test still fails, document the actual cause and link the resulting issue instead.

Written by the indexing model from the issue text.

Assessment

Tech stack
json
Domain
testing-qa, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.