asyncapi / asyncapi/website

Optimize directory traversal in check-edit-links script

Open
#3,586 34 comments 0 reactions 0 assignees View on GitHub
area/typescript enhancement stale
Dominant language
TypeScript
Stars
716
Forks
1.2k
Avg merge
1d 12h
Merged PRs (30d)
35

Description

## Background
Currently, the check-edit-links script reads the entire directory tree synchronously in the `generatePaths` function. For large documentation repositories, this could lead to performance issues.

## Proposed Solution
Consider using a streaming directory reader like readdir-enhanced or implementing a generator-based solution:

```javascript
async function* walkDirectory(dir) {
const files = await fs.readdir(dir);
for (const file of files) {
const filePath = path.join(dir, file);
const stats = await fs.stat(filePath);
if (stats.isDirectory()) {
yield* walkDirectory(filePath);
} else {
yield filePath;
}
}
}
```

## Benefits
- Memory efficient for large documentation repositories
- Better performance characteristics
- More modern async/await approach

Related PR discussion: https://github.com/asyncapi/website/pull/3557#discussion_r1921501710

Contributor guide

Open the contributing guide

Research direction

Locate the check-edit-links script and inspect the generatePaths function, then compare its current traversal with the related PR discussion. Run the script against a large documentation tree to establish current behavior and performance; done means link checking still works while traversal avoids loading the entire tree at once.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
performance, tooling
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.