Support jsdoc output larger than available RAM
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 290
- Forks
- 80
- PR merge metrics
- No merged PRs in 30d
Description
Incrementally read the JSON using something like https://pypi.python.org/pypi/ijson/ (which looks a little unusual (strange docstring formatting, needless pass use, implements its own hand-written JSON parser) but might be okay). Then keep just the indices (app._sphinxjs_doclets_by_path and app._sphinxjs_doclets_by_class) in RAM, and push the rest out to disk. In any case, we'll want to stop storing the doclets themselves in the indices and replace them with some sort of references.
Crazy brainstorming:
- Replace each doclet with just begin/end byte offsets where they occur in the jsdoc-emitted file, and read them in as needed, reparsing each time. Pro: no need to write all the doclets twice.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing how the jsdoc-emitted JSON populates app._sphinxjs_doclets_by_path and app._sphinxjs_doclets_by_class, and review the proposed ijson approach. Done means supporting output larger than available RAM while keeping only references or indices in memory and retrieving doclets as needed; the issue does not name a test or source file.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100