knitr::spin() result not cached, called 3 times per .R file during index build

Open
#14,228 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
55/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
typescript
Domain
performance

Research direction

Start in src/execute/rmd.ts at markdownFromKnitrSpinScript and its existing TODO, then trace the three calls from readBaseInputIndex described in the issue. Confirm the same .R file is processed repeatedly and verify that per-file caching reduces duplicate Rscript.exe spawns while preserving the spin markdown output.

Written by the indexing model from the issue text.

Description

engines-knitr enhancement performance

markdownFromKnitrSpinScript spawns a fresh Rscript.exe process each time it is called, and readBaseInputIndex triggers it 3 times for each .R spin script:

  1. renderFormats → resolveFullMarkdownForFile → markdownForFile → spin
  2. projectFileMetadata → markdownForFile → spin
  3. engine.partitionedMarkdown → spin

https://github.com/quarto-dev/quarto-cli/blob/5b88d3dfe1519179e2e8753c44e058a11ab62ff3/src/execute/rmd.ts#L581-L600

There is already a TODO in the code:

https://github.com/quarto-dev/quarto-cli/blob/5b88d3dfe1519179e2e8753c44e058a11ab62ff3/src/execute/rmd.ts#L584-L585

The spin result (markdown output) does not change between calls within the same render, so caching per file would eliminate 2 of the 3 R process spawns. In a project with many .R spin scripts, this adds up significantly (the reporter in #14225 measured 40s vs 3s).

Repro

https://github.com/byzheng/quarto-metadata-files-test

Context

Surfaced while investigating #14225. The spin caching issue is independent of the cache invalidation bug also reported from that investigation.

Dominant language
JavaScript
Stars
6k
Forks
458
Avg merge
1d 9h
Merged PRs (30d)
41

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from quarto-dev/quarto-cli

All issues in quarto-dev/quarto-cli

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.