getodk / getodk/web-forms

Consider partial consolidation of engine tests

Open
#100 0 comments 0 reactions 0 assignees View on GitHub
test-automation xforms-engine
Dominant language
TypeScript
Stars
38
Forks
22
PR merge metrics
No merged PRs in 30d

Description

As work on #25 progresses, it's been in the back of my mind that it will introduce a split between:

- the large volume of tests ported from JavaRosa, in the `scenario` package
- a currently smaller, but meaningful set of tests in the `xforms-engine` package

Conceptually, the split will likely eventually place integration tests in the former, and unit tests in the latter. I was initially tempted to open this issue titled "Consider consolidation of engine tests in `scenario` package", but this conceptual split occurred to me and it gave me pause on how we might want to approach such consolidation.

**Why consolidate anything?**

There are currently two disparate packages, with two disparate sets of `scripts` tasks which run different suites of tests for the same set of functionality. It's already fussy running commands across packages, and there's a strong incentive to run all engine-focused tests at once when working on engine functionality.

**Why _not_ consolidate all engine tests in `scenario`?**

Moving unit tests out of the `xforms-engine` package would either be severely limiting in terms of access to testable units, or impose a great deal of pressure to expose implementation details in client interfaces that we'd prefer to keep package-private.

**Why not consolidate all engine tests in `xforms-engine`? (Why a separate `scenario` package at all?)**

When I originally proposed creation of the package, it was mainly a "keep ourselves honest" measure: if the ported JavaRosa `Scenario` _is a client_, a separate package boundary is a good way to ensure that it has the same capabilities and restrictions as any other client.

I still think that's a good principle. Moreover, after quite a bit of work on #25, it's become clear that there's quite a lot of `Scenario`-specific implementation needed to support the actual tests as ported from JavaRosa, and none of that feels like it belongs organizationally within the engine itself.

**Options**

- Easiest lift: combine CLI invocations. Single-run commands would work similarly to other `npm-run-all` usages, watch commands would perhaps use a primitive mechanism for invoking task parallelism (like `&`).
- Worth considering: [Vitest workspaces](https://vitest.dev/guide/workspace) or some other purpose-build mechanism for orchestrating these test tasks across packages.
- ? (Open to other suggestions! I do consider it a high priority to accommodate both single-run and watch mode)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the test packages named in the issue: scenario and xforms-engine, along with their separate scripts tasks. Compare the existing npm-run-all approach with Vitest workspaces and the needs raised by #25. Done means an agreed consolidation or orchestration plan that supports both single-run and watch modes.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
build-system, testing
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.