Node.js: expand built-in module coverage (breadth + depth)
- Dominant language
- C#
- Stars
- 154
- Forks
- 4
- Avg merge
- 2h 46m
- Merged PRs (30d)
- 189
Description
**Standing task.** Iteratively grow SharpTS's Node.js compatibility surface — both **breadth** (modules we don't have at all) and **depth** (gaps inside modules we do). Supersedes epic #1096, whose two children (#1147 spike, #1148 assessment) both completed and produced the migration guidance now carried in §3.
## Where we stand
35 module specifiers are registered in `BuiltInModuleRegistry`:
```
assert async_hooks buffer child_process cluster crypto dgram dns dns/promises
events fs fs/promises http https net os path perf_hooks process querystring
readline stream stream/promises stream/web string_decoder timers timers/promises
tls tty url util vm worker_threads zlib
```
Full per-module surface detail is in STATUS.md §15. Two implementation strategies coexist:
| Implementation | Modules |
|---|---|
| **TypeScript stdlib** (`stdlib/node/*.ts`, embedded as resources) | `assert`, `async_hooks`, `events`, `fs`, `fs/promises`, `os`, `path`, `perf_hooks`, `process`, `querystring`, `readline`, `string_decoder`, `timers`, `timers/promises`, `tty`, `url`, `util`, `zlib` |
| **C# / IL** (dual hand-written: interpreter impl + IL emitter) | `crypto`, `stream`, `stream/promises`, `stream/web`, `buffer`, `http`, `https`, `net`, `tls`, `dgram`, `cluster`, `child_process`, `vm`, `dns`, `dns/promises`, `worker_threads` |
---
## 1. Breadth — modules we don't have at all
Roughly by expected value. Each is its own PR (or short series) with interpreter↔compiled parity and dual-mode tests.
**High value:**
- [ ] **`module`** — `createRequire`, `builtinModules`, `isBuiltin`, `register`, `syncBuiltinESMExports`. Small surface, and `builtinModules`/`isBuiltin` are exactly the introspection real packages use to feature-detect a runtime. Good first pick.
- [ ] **`v8`** — `serialize`/`deserialize` (structured clone over the wire), `getHeapStatistics`, `getHeapSpaceStatistics`, `setFlagsFromString`. The serialize pair is the useful half and we already have structured-clone machinery from `worker_threads`.
- [ ] **`diagnostics_channel`** — `channel`, `hasSubscribers`, `subscribe`/`unsubscribe`, `tracingChannel`. Pure in-process pub/sub, no host I/O — a natural TS-facade module with no primitive seam at all.
- [ ] **`readline/promises`** — promise-flavored `question`/`Interface`. We already have `readline` as a TS facade; this is the `fs`→`fs/promises` derivation pattern.
- [ ] **`stream/consumers`** — `text`, `json`, `arrayBuffer`, `blob`, `buffer`. Thin adapters over the existing `stream`.
- [ ] **`assert/strict`** — alias surface; `assert.strict` already exists.
**Larger / decide deliberately:**
- [ ] **`http2`** — big (HPACK, multiplexing, server push). Real work; worth its own epic when picked up.
- [ ] **`node:test`** — test runner + `test/reporters`. Substantial, but it's the module that makes SharpTS able to *run other projects' test suites*, which is a strong compatibility proof point.
- [ ] **`inspector`** / **`inspector/promises`** — needs a CDP surface; only meaningful alongside a debugging story.
- [ ] **`repl`** — we have a REPL; exposing it as a programmable module is a different (smaller) job than building one.
- [ ] **`sqlite`**, **`wasi`**, **`sea`**, **`trace_events`** — niche; pick up on demand.
**Deprecated — implement only if a real dependency needs them:** `punycode`, `domain`, `constants`, `sys`.
**Cheap alias/submodule specifiers worth checking:** `path/posix`, `path/win32`, `util/types`, `console` as a module form — the underlying surfaces already exist as properties; these may be registry entries rather than implementations.
## 2. Depth — gaps inside modules we already ship
Every module in STATUS.md §15 is marked ✅, but several carry documented ceilings and deviations. These are the honest compatibility gaps:
- [ ] **`crypto`** — EdDSA / x25519 / x448 unsupported (.NET 10 BCL ceiling, clear error). Compiled-deferred: KeyObject jwk/der import + `equals`, `ECDH.convertKey`, X509 `checkEmail`/`toLegacyObject`.
- [ ] **`process`** — `'unhandledRejection'`/`'rejectionHandled'` are interpreter-only (compiled deferred); POSIX `getuid`/`setgid`/etc. compiled-deferred. `--standalone` ceiling: `ppid` → 0.
- [ ] **`dns`** — compiled resolver callbacks run inline-sync (documented deviation); `Resolver.cancel()` is prompt only in the interpreter.
- [ ] **`worker_threads`** — `stdin`/`stdout`/`stderr` and `resourceLimits` options unsupported (workers share the parent's console).
- [ ] **`cluster`** — workers are in-process threads, so `worker.process.pid` is the host pid. Compiled `fork` runs the entry script interpreted (requires SharpTS.dll co-located + the entry `.ts` deployed); `--standalone` throws.
- [ ] **`path`** — `path.win32.isAbsolute('/foo')` returns `false`; Node returns `true` for any leading separator.
- [ ] **`dgram`** — udp6 source-specific membership throws (no portable .NET mapping).
**A good recurring exercise:** pick a widely-used npm package with no native deps, run its test suite under SharpTS, and file what breaks. That finds real gaps in a way a module checklist can't.
## 3. Strategy — TS facade over thin `primitive:*` seam
The `fs`/`os`/`process`/`zlib` migrations proved the pattern: the TypeScript facade owns the Node-shape surface, and only genuinely host-bound work sits behind a narrow `primitive:*` seam. **The `zlib` spike measured −823/+535 lines** — constants, error codes, all 11 async callback forms, and argument shuffling collapsed from two implementations into one shared by both modes. Compiled perf was flat within noise, `zlib.constants` access got **67× faster** compiled, and a latent interp↔compiled async divergence fell out in the wash.
**Prefer this shape for every new module.** A new module written as a TS facade is written once, not twice.
The #1148 assessment graded the remaining dual-C# cluster, and its recommended sequence still stands:
> **`dns` (S) → `net` partial (M) → `tls` partial (M) → `http` partial (L).**
> `crypto` opportunistic only (~5% glue). **`stream` is infeasible** (~1% glue — the classes *are* the API). **`dgram` skip** (~3% glue).
- [ ] **`dns` / `dns/promises` is the recommended next migration** — one-shot request/response shape, ~50% glue including the entire duplicated promise surface (`RuntimeEmitter.DnsPromises.cs` deletes outright), and it follows the exact `fs`/`fs/promises` derivation pattern already proven.
The win is real but uneven: facade migration pays where a module is logic-over-simple-primitives. Socket and stream modules have large *stateful* primitives, so the win there is the API-glue layer, not the core. **Drive it per-module; do not big-bang.** Current IL-emitter sizes for scale: `http` ~5,600 LOC, `stream` ~4,600, `dns` ~4,400, `net` ~4,300, `crypto` ~2,900, `tls` ~1,600 — each mirrored by a separate interpreter implementation.
## Working agreement for every PR here
- Green on `dotnet test`, `SharpTS.Test262`, and `SharpTS.TypeScriptConformance` — no baseline regression.
- **Interpreter and compiled output must be behaviourally identical.** Add dual-mode tests. Most bugs in this area have been interp↔compiled divergences, not missing features.
- **Standalone compiled DLLs must not gain a `SharpTS.dll` dependency** (CLAUDE.md "Standalone DLL Constraint"). If a module genuinely needs the runtime present, record it via `EmittedRuntime.RequireSharpTSRuntime(reason)` so the CLI co-locates SharpTS.dll — and only for paths a normal program actually reaches, never for pure-BCL features or graceful-fallback plumbing.
- Update STATUS.md §15 in the same PR — the table is the user-facing compatibility contract, and it drifts fast.
- Document deviations explicitly rather than marking a module ✅ and hiding the ceiling in a code comment.
Contributor guide
Research direction
Choose one unchecked module or documented gap, then read STATUS.md §15, BuiltInModuleRegistry, and the relevant stdlib/node or C# implementation. Run dotnet test, SharpTS.Test262, and SharpTS.TypeScriptConformance before and after the change. Done means interpreter and compiled behavior match, dual-mode tests pass, and STATUS.md §15 records the resulting surface or deviation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, node.js, typescript
- Domain
- backend, compilers, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100