frostney / frostney/GocciaScript

Add CI-native JUnit and TAP test reports

Open
#1,040 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

internal new feature runtime
Dominant language
Pascal
Stars
20
Forks
3
Avg merge
3d 4h
Merged PRs (30d)
45

Description

Summary

Add structured per-test results and built-in JUnit XML and TAP report formats to GocciaTestRunner.

Why

CI systems and editor tooling need standard machine-readable test reports. The current JSON envelope is useful for GocciaScript tooling but lacks the per-test records required to produce portable CI reports.

Current behavior

The testing library returns aggregate counts, a duration, suite names, and failed-test strings. The runner emits human output or a JSON envelope containing file-level aggregates; it has no normalized result for each test case and no JUnit or TAP output.

Expected behavior

  • Record each test case's file, suite ancestry, full name, status, duration, and failure details in a normalized result model.
  • Emit valid JUnit XML and TAP with explicit stdout/file destinations.
  • Represent skipped, todo, failed, timed-out, and load-error outcomes without losing existing diagnostics.
  • Keep exit status based on the test result rather than the selected report format.
  • Preserve existing human, JSON, and compact-JSON behavior.
  • Produce deterministic reports for serial and --jobs runs in interpreter and bytecode modes.

Scope notes

  • A custom reporter plugin API, HTML reports, browser UI, and provider-specific annotation APIs are out of scope.
  • Attempt records from future retry/repeat support should extend the normalized model rather than be flattened into strings.
  • Coverage formats and thresholds remain separate from test-result reporting.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating GocciaTestRunner and the existing human and JSON output paths, then compare serial and --jobs execution in interpreter and bytecode modes. Done means normalized per-test records, deterministic JUnit XML and TAP outputs, preserved existing formats and diagnostics, and exit status based on test results.

Written by the indexing model from the issue text.

Assessment

Domain
testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.