nickna / nickna/SharpTS

Node.js: expand built-in module coverage (breadth + depth)

Open
#1,282 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C#
Stars
154
Forks
4
Avg merge
2h 46m
Merged PRs (30d)
189

Description

**Standing task.** Iteratively grow SharpTS's Node.js compatibility surface — both **breadth** (modules we don't have at all) and **depth** (gaps inside modules we do). Supersedes epic #1096, whose two children (#1147 spike, #1148 assessment) both completed and produced the migration guidance now carried in §3.

## Where we stand

35 module specifiers are registered in `BuiltInModuleRegistry`:

```
assert async_hooks buffer child_process cluster crypto dgram dns dns/promises
events fs fs/promises http https net os path perf_hooks process querystring
readline stream stream/promises stream/web string_decoder timers timers/promises
tls tty url util vm worker_threads zlib
```

Full per-module surface detail is in STATUS.md §15. Two implementation strategies coexist:

| Implementation | Modules |
|---|---|
| **TypeScript stdlib** (`stdlib/node/*.ts`, embedded as resources) | `assert`, `async_hooks`, `events`, `fs`, `fs/promises`, `os`, `path`, `perf_hooks`, `process`, `querystring`, `readline`, `string_decoder`, `timers`, `timers/promises`, `tty`, `url`, `util`, `zlib` |
| **C# / IL** (dual hand-written: interpreter impl + IL emitter) | `crypto`, `stream`, `stream/promises`, `stream/web`, `buffer`, `http`, `https`, `net`, `tls`, `dgram`, `cluster`, `child_process`, `vm`, `dns`, `dns/promises`, `worker_threads` |

---

## 1. Breadth — modules we don't have at all

Roughly by expected value. Each is its own PR (or short series) with interpreter↔compiled parity and dual-mode tests.

**High value:**
- [ ] **`module`** — `createRequire`, `builtinModules`, `isBuiltin`, `register`, `syncBuiltinESMExports`. Small surface, and `builtinModules`/`isBuiltin` are exactly the introspection real packages use to feature-detect a runtime. Good first pick.
- [ ] **`v8`** — `serialize`/`deserialize` (structured clone over the wire), `getHeapStatistics`, `getHeapSpaceStatistics`, `setFlagsFromString`. The serialize pair is the useful half and we already have structured-clone machinery from `worker_threads`.
- [ ] **`diagnostics_channel`** — `channel`, `hasSubscribers`, `subscribe`/`unsubscribe`, `tracingChannel`. Pure in-process pub/sub, no host I/O — a natural TS-facade module with no primitive seam at all.
- [ ] **`readline/promises`** — promise-flavored `question`/`Interface`. We already have `readline` as a TS facade; this is the `fs`→`fs/promises` derivation pattern.
- [ ] **`stream/consumers`** — `text`, `json`, `arrayBuffer`, `blob`, `buffer`. Thin adapters over the existing `stream`.
- [ ] **`assert/strict`** — alias surface; `assert.strict` already exists.

**Larger / decide deliberately:**
- [ ] **`http2`** — big (HPACK, multiplexing, server push). Real work; worth its own epic when picked up.
- [ ] **`node:test`** — test runner + `test/reporters`. Substantial, but it's the module that makes SharpTS able to *run other projects' test suites*, which is a strong compatibility proof point.
- [ ] **`inspector`** / **`inspector/promises`** — needs a CDP surface; only meaningful alongside a debugging story.
- [ ] **`repl`** — we have a REPL; exposing it as a programmable module is a different (smaller) job than building one.
- [ ] **`sqlite`**, **`wasi`**, **`sea`**, **`trace_events`** — niche; pick up on demand.

**Deprecated — implement only if a real dependency needs them:** `punycode`, `domain`, `constants`, `sys`.

**Cheap alias/submodule specifiers worth checking:** `path/posix`, `path/win32`, `util/types`, `console` as a module form — the underlying surfaces already exist as properties; these may be registry entries rather than implementations.

## 2. Depth — gaps inside modules we already ship

Every module in STATUS.md §15 is marked ✅, but several carry documented ceilings and deviations. These are the honest compatibility gaps:

- [ ] **`crypto`** — EdDSA / x25519 / x448 unsupported (.NET 10 BCL ceiling, clear error). Compiled-deferred: KeyObject jwk/der import + `equals`, `ECDH.convertKey`, X509 `checkEmail`/`toLegacyObject`.
- [ ] **`process`** — `'unhandledRejection'`/`'rejectionHandled'` are interpreter-only (compiled deferred); POSIX `getuid`/`setgid`/etc. compiled-deferred. `--standalone` ceiling: `ppid` → 0.
- [ ] **`dns`** — compiled resolver callbacks run inline-sync (documented deviation); `Resolver.cancel()` is prompt only in the interpreter.
- [ ] **`worker_threads`** — `stdin`/`stdout`/`stderr` and `resourceLimits` options unsupported (workers share the parent's console).
- [ ] **`cluster`** — workers are in-process threads, so `worker.process.pid` is the host pid. Compiled `fork` runs the entry script interpreted (requires SharpTS.dll co-located + the entry `.ts` deployed); `--standalone` throws.
- [ ] **`path`** — `path.win32.isAbsolute('/foo')` returns `false`; Node returns `true` for any leading separator.
- [ ] **`dgram`** — udp6 source-specific membership throws (no portable .NET mapping).

**A good recurring exercise:** pick a widely-used npm package with no native deps, run its test suite under SharpTS, and file what breaks. That finds real gaps in a way a module checklist can't.

## 3. Strategy — TS facade over thin `primitive:*` seam

The `fs`/`os`/`process`/`zlib` migrations proved the pattern: the TypeScript facade owns the Node-shape surface, and only genuinely host-bound work sits behind a narrow `primitive:*` seam. **The `zlib` spike measured −823/+535 lines** — constants, error codes, all 11 async callback forms, and argument shuffling collapsed from two implementations into one shared by both modes. Compiled perf was flat within noise, `zlib.constants` access got **67× faster** compiled, and a latent interp↔compiled async divergence fell out in the wash.

**Prefer this shape for every new module.** A new module written as a TS facade is written once, not twice.

The #1148 assessment graded the remaining dual-C# cluster, and its recommended sequence still stands:

> **`dns` (S) → `net` partial (M) → `tls` partial (M) → `http` partial (L).**
> `crypto` opportunistic only (~5% glue). **`stream` is infeasible** (~1% glue — the classes *are* the API). **`dgram` skip** (~3% glue).

- [ ] **`dns` / `dns/promises` is the recommended next migration** — one-shot request/response shape, ~50% glue including the entire duplicated promise surface (`RuntimeEmitter.DnsPromises.cs` deletes outright), and it follows the exact `fs`/`fs/promises` derivation pattern already proven.

The win is real but uneven: facade migration pays where a module is logic-over-simple-primitives. Socket and stream modules have large *stateful* primitives, so the win there is the API-glue layer, not the core. **Drive it per-module; do not big-bang.** Current IL-emitter sizes for scale: `http` ~5,600 LOC, `stream` ~4,600, `dns` ~4,400, `net` ~4,300, `crypto` ~2,900, `tls` ~1,600 — each mirrored by a separate interpreter implementation.

## Working agreement for every PR here

- Green on `dotnet test`, `SharpTS.Test262`, and `SharpTS.TypeScriptConformance` — no baseline regression.
- **Interpreter and compiled output must be behaviourally identical.** Add dual-mode tests. Most bugs in this area have been interp↔compiled divergences, not missing features.
- **Standalone compiled DLLs must not gain a `SharpTS.dll` dependency** (CLAUDE.md "Standalone DLL Constraint"). If a module genuinely needs the runtime present, record it via `EmittedRuntime.RequireSharpTSRuntime(reason)` so the CLI co-locates SharpTS.dll — and only for paths a normal program actually reaches, never for pure-BCL features or graceful-fallback plumbing.
- Update STATUS.md §15 in the same PR — the table is the user-facing compatibility contract, and it drifts fast.
- Document deviations explicitly rather than marking a module ✅ and hiding the ceiling in a code comment.

Contributor guide

Open the contributing guide

Research direction

Choose one unchecked module or documented gap, then read STATUS.md §15, BuiltInModuleRegistry, and the relevant stdlib/node or C# implementation. Run dotnet test, SharpTS.Test262, and SharpTS.TypeScriptConformance before and after the change. Done means interpreter and compiled behavior match, dual-mode tests pass, and STATUS.md §15 records the resulting surface or deviation.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, node.js, typescript
Domain
backend, compilers, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.