JS conformance: close interpreted-mode Test262 parity deficits
- Dominant language
- C#
- Stars
- 154
- Forks
- 4
- Avg merge
- 2h 46m
- Merged PRs (30d)
- 189
Description
**Standing interpreter-parity task, refreshed 2026-08-14.** The original 73-test inventory has been completed and its 2,755 pinned parity cases are green, but compiler gains exposed a new set of interpreter deficits in the same committed Test262 subset. This issue deliberately tracks the live differential rather than preserving a stale hard-coded count in the title.
## Goal
Reduce the committed-subset interpreter-deficit set to zero:
```text
interpreted != Pass && compiled == Pass
```
Compiler-only deficits belong in #1374. Coverage growth, skip policy, and harness trust belong in #1280. Tests failing in both modes are general conformance work, not differential parity.
## Current state
Measured on `a4967cb5032a9a191eed8ade78db653bee2f804f` against Test262 corpus `d5e73fc8d2c663554fb72e2380a8c2bc1a318a33`. Two complete wide sweeps were intersected to remove pass-status flakes; the counts below are restricted back to the 11,384 paths in `config/subset.json`.
| Mode | Pass | Executed | Skipped | Pass rate |
|---|---:|---:|---:|---:|
| Interpreted | 9,706 | 10,505 | 879 | 92.4% |
| Compiled | 9,780 | 10,505 | 879 | 93.1% |
Stable pass/non-pass differential:
| Direction | Count | Scope |
|---|---:|---|
| interpreted non-pass → compiled Pass | **81** | **This issue** |
| interpreted Pass → compiled non-pass | 5 | #1374 (one is also a pooled-run isolation suspect) |
The two interpreted runs differed on only 3/47,159 wide-sweep buckets; 81 committed-subset interpreter deficits were present in both runs.
## Remaining stable interpreter deficits (81)
| Count | Area | Main sub-clusters |
|---:|---|---|
| **48** | Promise | `allSettled` 13, `any` 11, `all` 9, `race` 8, constructor/static metadata 7 |
| **24** | Object | `defineProperty` 11, `assign` 4, prototype branding 2, descriptors/extensibility/prototype/groupBy tail 7 |
| **5** | Array | `from` 2, `isArray` with Proxy 2, `toSorted` comparator validation 1 |
| **2** | JSON | reviver define/create property semantics |
| **2** | RegExp | word-class escape semantics 1, `Symbol.matchAll` IsRegExp lookup count 1 |
Why the count grew after completing the old 73: Track A is a moving differential. Several tests that previously failed in both modes now pass in compiled mode after the #1279 compiler-parity batches, revealing interpreter work that was not visible in the prior pass/non-pass comparison.
## Suggested order
1. [ ] Promise capability/combinator sequencing and generated resolve/reject function metadata (48).
2. [ ] Object descriptor, assignment, extensibility, and primitive-brand semantics (24).
3. [ ] Array `from` mutation boundaries, Proxy-aware `isArray`, and comparator validation (5).
4. [ ] JSON reviver property creation/definition semantics (2).
5. [ ] RegExp word-class and `Symbol.matchAll` lookup-count singletons (2).
6. [ ] Regenerate both committed baselines together and post the final differential.
Work by shared interpreter abstraction and pin representative cases. Do not special-case individual Test262 filenames.
## Completed work from the previous inventory
- [x] The original 73-test pinned inventory and its follow-on compiled parity set are green: `Issue1279ParityTests` passes 2,755/2,755.
- [x] Shared object/property descriptor behavior, primitive/string/RegExp protocols, JSON sequencing, globalThis exposure, inherited numeric coercion, and callable prototype lookup were repaired.
- [x] The full solution builds with zero warnings/errors and both IL verification passes.
- [x] Wide sweeps now persist gitignored per-mode path snapshots and automatically generate `wide-sweep-report.md` (`a4967cb5`).
## Reproduction
Run both modes from the same SharpTS commit and Test262 corpus revision:
```powershell
$env:SHARPTS_TEST262_WIDE_SWEEP = '1'
$env:SHARPTS_TEST262_WORKERS = '8'
dotnet test SharpTS.Test262\SharpTS.Test262.csproj `
--filter FullyQualifiedName=SharpTS.Test262.Test262InterpretedTests.InterpretedBaseline
dotnet test SharpTS.Test262\SharpTS.Test262.csproj `
--filter FullyQualifiedName=SharpTS.Test262.Test262CompiledTests.CompiledBaseline
```
The ignored path-level artifacts are written to `SharpTS.Test262/wide-sweep-baselines/`, with the differential at `SharpTS.Test262/wide-sweep-report.md`.
For the committed subset baseline contract, regenerate both modes together only when intentionally accepting the result changes:
```powershell
$env:SHARPTS_TEST262_UPDATE_BASELINE = '1'
dotnet test SharpTS.Test262\SharpTS.Test262.csproj
```
## Acceptance
- [ ] Stable Track A is zero within `config/subset.json` across two consecutive pooled runs.
- [ ] Both committed baselines are regenerated together from the same SharpTS commit and Test262 corpus revision.
- [ ] No new Test262 regressions in either mode.
- [ ] Focused parity tests pin each repaired shared abstraction.
- [ ] The final before/after histogram is posted.
Contributor guide
Research direction
Start with the two SharpTS.Test262 baseline commands and config/subset.json, using the reported differential to identify a representative failing case. Group the 81 deficits by shared interpreter abstraction rather than by Test262 filename, and add focused parity coverage for each repaired area. Done means two consecutive pooled runs report zero interpreted-only deficits, both baselines are regenerated together, and no regressions remain.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, typescript
- Domain
- compilers, testing-qa
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100