quarto-dev / quarto-dev/quarto-r

quarto::render() fails in parallel il project.render is "*.qmd"

Open
#224 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
R
Stars
161
Forks
29
Avg merge
3h 18m
Merged PRs (30d)
1

Description

Bug description

Rendering in parallel fails if project.render is "*.qmd" in _quarto.yaml

Reprex

Let's assume the following _quarto.yaml:

project:
  render:
    - "*.qmd"

Which is found on the doc at https://quarto.org/docs/projects/quarto-projects.html#render-targets

If I create a series of qmd and tries to render them in parallel using {future}, it fails :

options(
  future.rng.onMisuse = "ignore"
)
future::plan(future::multisession)

for (i in 1:50){
  write(file = sprintf("test%s.qmd", i),
        sprintf("---
title: 'test%s'
---

## Quarto

Quarto enables you to weave together content and executable code into a finished presentation. To learn more about Quarto presentations see <https://quarto.org/docs/presentations/>.
", i)
  )
}

list.files(pattern = "\\.qmd$", full.names = TRUE) |>
  furrr::future_map(
    .progress = TRUE,
    \(x){
      quarto::quarto_render(
        x,
        quiet = FALSE
      )
    }
  )

With the following error:

ERROR: NotFound: No such file or directory (os error 2): stat '/Users/colinfay/git/poubelle/multirenderquarto/test15_files/mediabag'

Stack trace:
    at Object.statSync (ext:deno_fs/30_fs.js:425:3)
    at createWalkEntrySync (file:///Applications/quarto/bin/quarto.js:2028:23)
    at walkSync (file:///Applications/quarto/bin/quarto.js:2068:15)
    at walkSync.next (<anonymous>)
    at advanceMatch (file:///Applications/quarto/bin/quarto.js:2191:20)
    at advanceMatch.next (<anonymous>)
    at expandGlobSync (file:///Applications/quarto/bin/quarto.js:2207:24)
    at expandGlobSync.next (<anonymous>)
    at expandGlobs (file:///Applications/quarto/bin/quarto.js:4972:24)
    at resolveGlobs (file:///Applications/quarto/bin/quarto.js:5049:26)
Error in (function (.x, .f, ..., .progress = FALSE)  : 
  ℹ In index: 3.
Caused by error in `quarto::quarto_render()`:
✖ Error running quarto cli.
Caused by error:
! System command 'quarto' failed
Calls: <Anonymous> ... resolve.list -> signalConditionsASAP -> signalConditions
Execution halted

Notes

  • If I modify the yaml on the fly and add all the qmd names, it works :
y <- yaml::read_yaml(
  "_quarto.yaml"
)
old_render <- y$project$render
y$project$render <- list.files(pattern = "qmd")
yaml::write_yaml(y, "_quarto.yaml")

list.files(pattern = "\\.qmd$", full.names = TRUE) |>
  furrr::future_map(
    .progress = TRUE,
    \(x){
      quarto::quarto_render(
        x,
        quiet = FALSE
      )
      return(x)
    }
  )
  
y$project$render <- old_render
yaml::write_yaml(y, "_quarto.yaml")
  • Removing the _quarto.yml works

  • A _quarto.yml with other content will work

format:
  html

Session information

% Rscript -e "packageVersion('quarto')"
[1] ‘1.4.4’
% quarto --version
1.6.40

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the supplied R future::multisession and furrr::future_map() reproducer, using quarto::quarto_render() and the shown _quarto.yaml with project.render: "*.qmd". Inspect the R interface entry point and Quarto CLI glob resolution. Done means parallel rendering succeeds without manually expanding the render list.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
cli, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.