nickna / nickna/SharpTS

Parser: complete expression and type-node source-span provenance

Open
#1,391 0 comments 0 reactions 0 assignees View on GitHub
deferred enhancement
Dominant language
C#
Stars
154
Forks
4
Avg merge
2h 46m
Merged PRs (30d)
189

Description

Parent epic: #1390. Deferred from #1306.

## Goal

Give every source-backed expression and type node an exact, half-open UTF-16 `SourceSpan`, and preserve or hide that provenance through parser/compiler transforms. This is enabling infrastructure for position-sensitive editor features; it must not become a second syntax tree or change AST record equality.

## Current state

`SourceDocument`/`SpanTable` cover declarations, executable statements, and transformed statement provenance. Tokens already carry exact offsets, so existing lexical navigation works. Broad `Expr` and `TypeNode` coverage remains incomplete, and parts of the type-AST migration still carry string forms plus optional node twins.

## Work

- [ ] Inventory every concrete `Expr`, `TypeNode`, and source-addressable auxiliary node (object/type members, tuple elements, signatures, property keys) and classify it as source-backed, synthesized-with-provenance, or intentionally hidden.
- [ ] Add parser helpers that record a production from its first consumed token through its true final token without duplicating offset arithmetic across productions.
- [ ] Instrument all expression productions, including assignment/conditional/binary/unary chains, calls/new/member/index access, arrows/functions/classes, object/array/template literals, JSX/TSX, assertions/satisfies, dynamic import, optional chaining, and computed keys.
- [ ] Instrument all type productions, including qualified/generic references, function/constructor types, unions/intersections/conditionals, mapped/indexed/keyof/infer types, object members/signatures, tuples, predicates, template-literal types, import types, and heritage/type-argument nodes.
- [ ] Preserve offsets across token rescans and split `>>`/`>>>` generic closers.
- [ ] Audit every transform/lowering that replaces or synthesizes expressions/types. Use `CopySpan` for a source-equivalent replacement and `MarkHidden` for scaffolding; never attribute a synthetic operation to an unrelated nearby expression.
- [ ] Ensure position lookup can select the narrowest node deterministically when parent/child spans share boundaries.
- [ ] Complete required type-node twins rather than manufacturing spans for legacy string-only annotations.
- [ ] Document coverage rules and the contract for future AST node additions.

## Verification

- [ ] Add a reflection/visitor-driven coverage test that walks representative parsed ASTs and fails when a source-backed expression/type node has no span.
- [ ] Assert every recorded span is within its `SourceDocument`, is half-open, and is contained by its source-backed parent where grammar nesting requires containment.
- [ ] Add exact-offset tests for multiline expressions/types, Unicode surrogate pairs, comments/trivia, templates, JSX/TSX, decorators, optional chains, nested generics with split closers, and parser recovery.
- [ ] Add transform tests for destructuring, var hoisting, generator-arrow lifting, nested-function lifting, JSX lowering, async/generator rewrites, and other expression-producing lowerings; synthetic nodes must be hidden or explicitly traceable.
- [ ] Verify multi-file and virtual/embedded stdlib documents never exchange span provenance.
- [ ] Existing debugger sequence-point and LSP navigation suites remain green.
- [ ] Add/extend parser benchmarks and record before/after wall time and allocations on small files plus a representative multi-module project; investigate and document any regression above 10%.
- [ ] Full unit, Test262, and TypeScript conformance validation is green.

## Success criteria

The issue is complete when coverage tests make missing provenance on any supported source-backed expression/type production a regression, transform provenance is explicit, position lookup is deterministic, and the measured cost is acceptable and documented.

## Non-goals

- Emitting expression-level PDB sequence points merely because spans exist.
- Giving ordinary spans to compiler-only scaffolding.
- Implementing an editor feature in this infrastructure issue.

Contributor guide

Open the contributing guide

Research direction

Start with SourceDocument/SpanTable and the parser entry points for Expr and TypeNode, then inventory the concrete nodes and transforms named in the issue. Add the reflection/visitor-driven coverage and exact-offset tests, and verify debugger/LSP, conformance, benchmark, and multi-file provenance checks. Done means supported source-backed nodes have deterministic spans, synthetic nodes are hidden or traceable, and measured cost is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, typescript
Domain
compilers, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.