Adaptor docs sometimes pulls down too much data (and so is slow)
Nobody has claimed this yet.
- Dominant language
- Elixir
- Stars
- 296
- Forks
- 86
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 50
Description
This issue doesn't technically belong here, it's kind of a cross-repo thing. But it's impact is here and it can be managed more effectively from here I think.
The Issue
When loading documentation for some pre-monorepo adaptors, the client can download far more information than it needs. language-common@1.7.1 for example, downloads hundreds of files (!!), meaning it takes docs a lot longer to generate than it should.
The problem is:
adaptor-docsaskskit/describe-packagefor metadata about a packagedescribe-packageloads thepackage.jsonmanifest for that package, and downloads every d.ts file listed in thefilesarray- Some (all?) adaptor package.jsons list files from its node_modules - files which surely shouldn't be listed.
It is probably worth examining the impact a bit more before deciding the severity of this. To be fair I think it's quite a simple fix.
Solutions
Probably the best fix is in kit/describe-package. When fetching a file listing from npm, we simply ignore anything that starts with node_modules.
An alternative is to wait for the doc builder to move out of the client and into the server (see #492). With caching in place the impact of this is greatly reduced. We may consider fixing describe-package but it'll be less critical.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with adaptor-docs and kit/describe-package, then inspect how the package.json files array is used when fetching the language-common@1.7.1 file listing. Confirm the excess downloads and determine whether node_modules entries can be ignored there; done means documentation generation no longer fetches those unnecessary files.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js
- Domain
- documentation, performance, tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100