stdlib-js / stdlib-js/stdlib

[RFC]: Extend stdlib's doctesting approach to C examples

Aperta
#13,194 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
Accepted C RFC Tools
Lingua principale
JavaScript
Stelle
6k
Fork
1.3k
Merge medio
1g 3h
PR unite (30g)
611

Descrizione

## Description

This RFC proposes a suite of packages under `@stdlib/_tools/doctest/c` implementing an automated `extract → instrument → compile → run → compare` pipeline for testing C code examples embedded in `src/*.c` source files and `README.md` files.

---

## Proposed Changes

### Package layout

```
@stdlib/_tools/doctest/c/ ← orchestrator (bin/cli)
├── extract/
├── instrument/
├── compile/
├── run/
└── compare/
```

Make targets follow the existing `doctest` namespace in `tools/make/lib/doctest/c/doctest-c.mk`.

---

### Sub-package API

#### `@stdlib/_tools/doctest/c/extract`

```js
extract( filePath, mode ) → Array
// filePath: string — absolute path to a .c source file or README.md
// mode: 'src' | 'markdown'
// returns an array of self-contained block objects, one per testable example
/* block = {
file: string,
blockIndex: number, ← 0-based, helps segregating example blocks from same file
includes: Array<{ text: string, line: number }>, ← #include lines hoisted for this block
staticHelpers: Array<{ text: string, line: number }>, ← array of hoisted internal static functions called by this block
fileScope: Array<{ text: string, line: number }>, ← pre-function declarations + function definitions for this block (to be hoisted outside main()) (ex, @stdlib/ndarray/base/function-object)
body: Array<{ text: string, line: number }>, ← procedural lines for main() for this block
annotations: Array<{ text: string, line: number }> ← extracted annotation values (e.g., "~1.571", "true") from // returns comments
}
*/
```

Parses `@example` Doxygen blocks (`src` mode) or ` ```c ``` ` fenced blocks (`markdown` mode). Each block is independent.
*Note*: Contiguous lines are grouped into a single text block. The `line` property indicates the starting line number of that contiguous block

#### `@stdlib/_tools/doctest/c/instrument`

```js
instrument( block ) → { tmpFile, annotations }
// block: block object from extract()
// tmpFile: absolute path to the generated .c file written to os.tmpdir()
// annotations: Array<{ annotation, type, varName, file, line }>
```

Wraps the block into a compilable `.c` file: injects necessary stdlib and standard library headers, hoists static helpers and file-scope declarations, wraps procedural code in `main()`, and emits a type-aware `printf` per `// returns` annotation. Injects `#line N "original/file.c"` directives so all compiler errors reference the original file and line, not the generated tmp file.
For each annotation, type inference runs via a inferType helper that backward-scans the body lines accumulated up to that annotation to find the preceding variable declaration.

#### `@stdlib/_tools/doctest/c/compile`

```js
compile( tmpFile ) → { ok, binary, stderr }
// tmpFile: path from instrument()
// binary: path to the compiled executable (in os.tmpdir())
// stderr: raw compiler output (already line-mapped to original file via #line)
```

Resolves include paths via `manifest.json` using `@stdlib/utils/library-manifest`. Handles supplemental headers referenced inside a snippet but absent from the package's core manifest dependencies.

#### `@stdlib/_tools/doctest/c/run`

```js
run( binary, opts ) → { ok, stdout, stderr, timedOut }
// binary: path from compile()
// opts: { timeout: number } — default 15000 ms
```

Executes the compiled binary via `spawnSync` with a configurable timeout.

#### `@stdlib/_tools/doctest/c/compare`

```js
compare( stdout, annotations ) → Array<{ pass, got, expected, file, line }>
// stdout: raw stdout string from run()
// annotations: annotation metadata array from instrument()
```

Normalizes tokens (`inf`/`-inf` → `±Infinity`, `nan` → `NaN`, trailing `f` stripped) before comparison. Uses `Object.is` for `-0` detection and delegates `~`-prefixed approximate values to `@stdlib/_tools/doctest/compare-values` and direct comparison for ints and bools.

---

### Block Skip Rules

In `src/*.c` files, a trailing comment on the function definition opts the `@example` blocks of that function out:

```c
// Skips @example blocks of that function:
static double foo( double x ) { // stdlib-c-doctest disable
...
}

// Skip all remaining @example blocks in the file:
// stdlib-c-doctest disable-file
```

In `README.md` files, the existing `` comment before a fenced block opts it out. `napi` packages (paths containing `/napi/`) are excluded automatically by the orchestrator.

---

### CLI

Invoked by make targets and pre-commit hooks, one file at a time:

```bash
node lib/node_modules/@stdlib/_tools/doctest/c/bin/cli.js \
--mode=src \
--file=lib/node_modules/@stdlib/math/base/special/sin/src/main.c \
[--verbose] [--timeout=]
```

| Flag | Description |
|---|---|
| `--mode` | `src` or `markdown` |
| `--file` | File to process |
| `--verbose` | Print compiler output and instrumentation trace |
| `--timeout` | Per-binary execution timeout in ms (default: 15000) |

### Output Format

Compiler errors are surfaced directly from the compiler's own output, which are already mapped to the original file via #line directives. Annotation mismatches and pipeline errors (compare stage) are formatted by the orchestrator as `file:line:col: error: message [rule]`.

---

### Stretch Goals

- Mutated scalar annotation: `// out => 0.5`
- Flat array annotation: `// x => [ 1.0, 2.0, 3.0 ]`
- Complex number annotation: `// returns [re, im]`

### Related Issues

Related issues [#G96](https://github.com/stdlib-js/google-summer-of-code/issues/96) , #1378.

---
### Questions

1. **Include path resolution:** Should supplemental dep resolution remain in the doctest tool, or should a `docs` task config be added to `manifest.json` so `library-manifest` handles it fully?

2. **Static helper handling:** Should internal `static` helpers be hoisted exactly into the tmp file, or promoted to namespaced internal symbols (e.g., `stdlib_internal_*`) via a restricted header?

3. **Tool structuring/naming:** Any feedback on the sub-package naming or placement under `@stdlib/_tools/doctest/c/`?

4. **Skip rule scope for `@example` blocks:** When `// stdlib-c-doctest disable` is placed on a function definition, it skips all `@example` blocks for that function. However, since a function can have multiple example blocks, should there be a way to skip specific blocks (e.g., using an index-based approach)?

### Other

No.

### Checklist

- [x] I have read and understood the [Code of Conduct](https://github.com/stdlib-js/stdlib/blob/develop/CODE_OF_CONDUCT.md).
- [x] Searched for existing issues and pull requests.
- [x] The issue name begins with `RFC:`.

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Start by reading the existing doctest targets in tools/make/lib/doctest/c/doctest-c.mk and the related issues #G96 and #1378. Review the proposed extract, instrument, compile, run, and compare boundaries, then resolve the include-path, static-helper, naming, and skip-rule questions before implementation; done means an agreed design for the C doctesting pipeline.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
c, javascript, node.js
Ambito
build-system, testing-qa, tooling
Tipo di issue
Funzionalità
Difficoltà
5/5
Tempo stimato
Più di una settimana
Stato di attività
Tranquilla
Chiarezza
Abbastanza chiara
Idoneità per principianti
28/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.