posit-dev / posit-dev/great-docs

Tools for building and debugging notebooks

Open
#353 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Type: ★ Enhancement
Dominant language
Python
Stars
262
Forks
18
Avg merge
1d 4h
Merged PRs (30d)
14

Description

For many big docsets, almost all the build time goes into executing the reference notebooks, e.g. user guide/tutorial pages. Quarto already provides freeze and cache options, but these are invalidated based on the notebook itself being updated, when frequently the notebooks need to be regenerated when the Python package itself changes.

In my experience, there are two important tasks that are hard to do with Quarto:

  1. Build all the notebooks in parallel rather than serially
  2. Check/debug the notebooks

Building in parallel should probably happen by default when great-docs build is run (or maybe great-docs build --parallel if that seems too risky). The second should execute each notebook, not updating the built docs or notebook cache, but summarizing any build errors.

Both are fortunately fairly straightforward. It would need more testing, but I've done a draft implementation in my quartopydoc fork -- see the code and the docs. (A lot of what I was trying to do with qpyd is what you've already built into Great Docs but about 100x better, so thank you thank you!)

If you want to include these features in Great Docs, I can work on a PR in the next month or two, depending on what would fit with your development schedule. Or if you want to borrow liberally from quartopydoc, that is of course 100% welcome too!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the great-docs build entry point and compare the proposed behavior with the quartodoc/qpyd code and qpynb CLI documentation linked in the issue. Investigate parallel notebook execution and a check/debug mode that reports errors without updating docs or the notebook cache. Done means both workflows are implemented and tested.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, python
Domain
build-system, documentation, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.