quarto-dev / quarto-dev/quarto-cli

Parallel creation of parameterized reports fails due to file conflicts

Open
#4,730 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement files
Dominant language
JavaScript
Stars
6k
Forks
458
Avg merge
1d 9h
Merged PRs (30d)
41

Description

Bug description

I'm trying to generate parameterized reports in parallel on Ubuntu 22.04 LTS using quarto=1.2.335. For this, I'm using the following test.qmd file:

---
title: paralell report generation fails
params:
  counter: 1
format:
  html:
    embed-resources: true
---

\```{r}
Sys.sleep(5)
\```

This is report number `r params$counter`

Generating both report out1.html and out2.html works fine when running sequentially:

quarto render test.qmd -P counter:1 --output out1.html
quarto render test.qmd -P counter:2 --output out2.html

However, when I try to run them in parallel I get:

quarto render test.qmd -P counter:1 --output out1.html &;
quarto render test.qmd -P counter:2 --output out2.html
# Error in readLines(con, warn = FALSE) : cannot open the connection
# Calls: .main ... partition_yaml_front_matter -> grep -> is.factor -> read_utf8 -> readLines

I suspect this is because both reports store their temporary files in test_files/. Wouldn't it make more sense to use the output file base as directory, because this should not conflict?

Checklist
  • Please include a minimal, fully reproducible example in a single .qmd file? Please provide the whole file rather than the snippet you believe is causing the issue.
  • Please format your issue so it is easier for us to read the bug report.
  • Please document the RStudio IDE version you're running (if applicable), by providing the value displayed in the "About RStudio" main menu dialog?
  • Please document the operating system you're running. If on Linux, please provide the specific distribution.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the provided test.qmd and run the two quarto render commands concurrently on Ubuntu 22.04 with quarto 1.2.335. Inspect how the parallel renders use test_files/ and reproduce the readLines failure; done means both out1.html and out2.html render successfully without conflicting temporary files.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, r
Domain
build-system, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.