Optimize directory traversal in check-edit-links script
- Dominant language
- TypeScript
- Stars
- 716
- Forks
- 1.2k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 35
Description
## Background
Currently, the check-edit-links script reads the entire directory tree synchronously in the `generatePaths` function. For large documentation repositories, this could lead to performance issues.
## Proposed Solution
Consider using a streaming directory reader like readdir-enhanced or implementing a generator-based solution:
```javascript
async function* walkDirectory(dir) {
const files = await fs.readdir(dir);
for (const file of files) {
const filePath = path.join(dir, file);
const stats = await fs.stat(filePath);
if (stats.isDirectory()) {
yield* walkDirectory(filePath);
} else {
yield filePath;
}
}
}
```
## Benefits
- Memory efficient for large documentation repositories
- Better performance characteristics
- More modern async/await approach
Related PR discussion: https://github.com/asyncapi/website/pull/3557#discussion_r1921501710
Contributor guide
Research direction
Locate the check-edit-links script and inspect the generatePaths function, then compare its current traversal with the related PR discussion. Run the script against a large documentation tree to establish current behavior and performance; done means link checking still works while traversal avoids loading the entire tree at once.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js
- Domain
- performance, tooling
- Issue type
- Refactor
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100