influxdata / influxdata/docs-v2
Migrate scripts with private YAML and JSON loaders to scripts/lib/file-operations.js
- Dominant language
- JavaScript
- Stars
- 82
- Forks
- 326
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 82
Description
## Problem
`scripts/lib/file-operations.js` exports shared `readJson` and `readYaml` helpers, but most Node scripts in the repo still parse files inline with `yaml.load(readFileSync(...))` or `JSON.parse(readFileSync(...))`, or carry a private `loadYaml`/`readYaml` copy. Each copy handles a missing or malformed file differently: some throw the raw `ENOENT`, some fall back to `{}` silently, one returns `null`. A malformed `data/products.yml` therefore fails with a different message, or does not fail at all, depending on which script reads it first.
`readYaml` was added on branch `claude/relaxed-hypatia-6m8hoh` (commit `cdf1af5`) when the release gate and products.yml schema checkers were written. It throws `cannot read ` or `cannot parse ` with the original error attached as `cause`, so callers can print the message as-is. The two new checkers use it; the older scripts do not.
## Scope
Replace inline `readFileSync` + `yaml.load` / `JSON.parse` with `readYaml` / `readJson` from `scripts/lib/file-operations.js` in:
**CI checkers (`.ci/scripts/`)**
- `detect-version-bump.js` (private `load()` that falls back to `{}`; decide whether the fallback is still wanted now that `check-release-gate.js` exists)
- `check-release-notes-version.js`
- `check-feedback-links.js`
**Build and helper scripts**
- `scripts/build-llms-txt.js`
- `scripts/build-llms-full-txt.js`
- `scripts/build-llm-markdown.js`
- `scripts/check-md-alternate-coherence.js`
- `scripts/fix-redoc-anchors.mjs`
- `scripts/lib/content-scaffolding.js` (`loadProducts`; keep its reshaping, replace only the read)
- `scripts/lib/provenance.js`
- `scripts/docs-cli/lib/config-loader.js` (YAML and JSON)
- `scripts/docs-cli/lib/api-doc-scanner.js`
- `scripts/docs-cli/commands/release-notes.js` (JSON)
- `helper-scripts/build-agent-instructions.js`
- `helper-scripts/influxdb3-plugins/port_to_docs.js`
- `cypress.config.js`
**API docs scripts (TypeScript, `api-docs/scripts/`)**
- `post-process-specs.ts` (private `loadYaml` returning `null` on failure)
- `test-post-process-specs.ts` (private `readYaml`)
- `generate-openapi-articles.ts`
- `openapi-paths-to-hugo-data/index.ts`
These are TypeScript and compiled to `api-docs/scripts/dist/`; importing a `.js` helper from `scripts/lib` may need a type declaration or a small TS wrapper. Do not edit `dist/` by hand; rebuild it with `yarn build:api-docs-scripts` (or the pre-commit `build-api-docs-scripts` hook).
## Out of scope
- `assets/js/**` (`node-shim.ts`, `product-mappings.ts`, `product-labels.test.mjs`): these are bundled for the browser and cannot import a Node `fs` helper.
- Test files that read their own fixtures (`scripts/__tests__/*.test.mjs`) unless a migration is trivial.
## Approach
1. One PR per group above so each can be reviewed and reverted on its own.
2. For each script, check whether it relied on a silent fallback (empty object, `null`) and decide whether to keep that behavior with an explicit `try/catch` at the call site, or let the clearer error propagate. Record the decision in the commit body.
3. Run the existing tests for each group (`node --test scripts/__tests__/...`, `node --test .ci/scripts/*.test.js`, `yarn test:build-llms-full`, and the api-docs test script) plus `yarn eslint` on changed files. `.ci/scripts/**/*.js` now has Node globals in `eslint.config.js`, so those files lint cleanly.
## Done when
- No script under `.ci/scripts/`, `scripts/`, `helper-scripts/`, or `api-docs/scripts/` (excluding `dist/`) defines its own YAML or JSON reader or calls `yaml.load(readFileSync(...))` / `JSON.parse(readFileSync(...))` directly. `rg -n "yaml\.load\(|JSON\.parse\(\s*(fs\.)?readFileSync" --glob '!node_modules/**' --glob '!**/dist/**' --glob '!assets/**'` returns only `scripts/lib/file-operations.js`.
- A malformed `data/products.yml` produces the same `cannot parse data/products.yml: ...` message from every script that reads it.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with scripts/lib/file-operations.js and compare its readJson/readYaml behavior with the listed scripts, especially .ci/scripts/detect-version-bump.js and the TypeScript files under api-docs/scripts/. Run the relevant node --test commands, yarn test:build-llms-full, yarn build:api-docs-scripts, and yarn eslint while migrating one group at a time. Done means the listed scripts use the shared readers, excluded paths remain untouched, and the rg check finds no remaining inline readers.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js, typescript
- Domain
- build-system, tooling
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100