nickna / nickna/SharpTS

JS conformance: close interpreted-mode Test262 parity deficits

Open
#1,279 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C#
Stars
154
Forks
4
Avg merge
2h 46m
Merged PRs (30d)
189

Description

**Standing interpreter-parity task, refreshed 2026-08-14.** The original 73-test inventory has been completed and its 2,755 pinned parity cases are green, but compiler gains exposed a new set of interpreter deficits in the same committed Test262 subset. This issue deliberately tracks the live differential rather than preserving a stale hard-coded count in the title.

## Goal

Reduce the committed-subset interpreter-deficit set to zero:

```text
interpreted != Pass && compiled == Pass
```

Compiler-only deficits belong in #1374. Coverage growth, skip policy, and harness trust belong in #1280. Tests failing in both modes are general conformance work, not differential parity.

## Current state

Measured on `a4967cb5032a9a191eed8ade78db653bee2f804f` against Test262 corpus `d5e73fc8d2c663554fb72e2380a8c2bc1a318a33`. Two complete wide sweeps were intersected to remove pass-status flakes; the counts below are restricted back to the 11,384 paths in `config/subset.json`.

| Mode | Pass | Executed | Skipped | Pass rate |
|---|---:|---:|---:|---:|
| Interpreted | 9,706 | 10,505 | 879 | 92.4% |
| Compiled | 9,780 | 10,505 | 879 | 93.1% |

Stable pass/non-pass differential:

| Direction | Count | Scope |
|---|---:|---|
| interpreted non-pass → compiled Pass | **81** | **This issue** |
| interpreted Pass → compiled non-pass | 5 | #1374 (one is also a pooled-run isolation suspect) |

The two interpreted runs differed on only 3/47,159 wide-sweep buckets; 81 committed-subset interpreter deficits were present in both runs.

## Remaining stable interpreter deficits (81)

| Count | Area | Main sub-clusters |
|---:|---|---|
| **48** | Promise | `allSettled` 13, `any` 11, `all` 9, `race` 8, constructor/static metadata 7 |
| **24** | Object | `defineProperty` 11, `assign` 4, prototype branding 2, descriptors/extensibility/prototype/groupBy tail 7 |
| **5** | Array | `from` 2, `isArray` with Proxy 2, `toSorted` comparator validation 1 |
| **2** | JSON | reviver define/create property semantics |
| **2** | RegExp | word-class escape semantics 1, `Symbol.matchAll` IsRegExp lookup count 1 |

Why the count grew after completing the old 73: Track A is a moving differential. Several tests that previously failed in both modes now pass in compiled mode after the #1279 compiler-parity batches, revealing interpreter work that was not visible in the prior pass/non-pass comparison.

## Suggested order

1. [ ] Promise capability/combinator sequencing and generated resolve/reject function metadata (48).
2. [ ] Object descriptor, assignment, extensibility, and primitive-brand semantics (24).
3. [ ] Array `from` mutation boundaries, Proxy-aware `isArray`, and comparator validation (5).
4. [ ] JSON reviver property creation/definition semantics (2).
5. [ ] RegExp word-class and `Symbol.matchAll` lookup-count singletons (2).
6. [ ] Regenerate both committed baselines together and post the final differential.

Work by shared interpreter abstraction and pin representative cases. Do not special-case individual Test262 filenames.

## Completed work from the previous inventory

- [x] The original 73-test pinned inventory and its follow-on compiled parity set are green: `Issue1279ParityTests` passes 2,755/2,755.
- [x] Shared object/property descriptor behavior, primitive/string/RegExp protocols, JSON sequencing, globalThis exposure, inherited numeric coercion, and callable prototype lookup were repaired.
- [x] The full solution builds with zero warnings/errors and both IL verification passes.
- [x] Wide sweeps now persist gitignored per-mode path snapshots and automatically generate `wide-sweep-report.md` (`a4967cb5`).

## Reproduction

Run both modes from the same SharpTS commit and Test262 corpus revision:

```powershell
$env:SHARPTS_TEST262_WIDE_SWEEP = '1'
$env:SHARPTS_TEST262_WORKERS = '8'
dotnet test SharpTS.Test262\SharpTS.Test262.csproj `
--filter FullyQualifiedName=SharpTS.Test262.Test262InterpretedTests.InterpretedBaseline
dotnet test SharpTS.Test262\SharpTS.Test262.csproj `
--filter FullyQualifiedName=SharpTS.Test262.Test262CompiledTests.CompiledBaseline
```

The ignored path-level artifacts are written to `SharpTS.Test262/wide-sweep-baselines/`, with the differential at `SharpTS.Test262/wide-sweep-report.md`.

For the committed subset baseline contract, regenerate both modes together only when intentionally accepting the result changes:

```powershell
$env:SHARPTS_TEST262_UPDATE_BASELINE = '1'
dotnet test SharpTS.Test262\SharpTS.Test262.csproj
```

## Acceptance

- [ ] Stable Track A is zero within `config/subset.json` across two consecutive pooled runs.
- [ ] Both committed baselines are regenerated together from the same SharpTS commit and Test262 corpus revision.
- [ ] No new Test262 regressions in either mode.
- [ ] Focused parity tests pin each repaired shared abstraction.
- [ ] The final before/after histogram is posted.

Contributor guide

Open the contributing guide

Research direction

Start with the two SharpTS.Test262 baseline commands and config/subset.json, using the reported differential to identify a representative failing case. Group the 81 deficits by shared interpreter abstraction rather than by Test262 filename, and add focused parity coverage for each repaired area. Done means two consecutive pooled runs report zero interpreted-only deficits, both baselines are regenerated together, and no regressions remain.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, typescript
Domain
compilers, testing-qa
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.