Pre-render scripts are run after Quarto checks for existing files when project type: book

Bug description

Related to #3879.

I am trying to write a Quarto book where the source files are organized into multiple subdirectories, but the output is "flattened". An example of the intended structure is below:

├── docs
│   ├── a.html
│   ├── b.html
│   └── index.html
├── index.qmd
├── post.R
├── pre.R
├── _quarto.yml
└── src
    ├── a
    │   └── a.qmd
    └── b
        └── b.qmd

After some googling, I decided to do this by running pre- and post-render scripts (files pre.R and post.R in the above intended directory structure). The pre-render script traverses the src folder and stores the relative paths into a csv file in ., as well as moves all files to the top-level directory (i.e., from ./path/to/file.qmd to ./file.qmd). The post-render script moves all the previously moved qmd files back into their original directories.

Since the move should happen pre-render, I've defined the document structure in my _quarto.yml file to match the file structure after moving the files (i.e., the flattened structure, where all files reside in the top-level directory). However, when calling quarto render, it throws an error of files not found in the top-level directory. In other words: Quarto is erroring before executing the pre-render script.

I've gotten this pipeline to work by manually running the pre-render script before rendering, then rendering and letting Quarto run the post-render script (as opposed to Quarto running both pre- and post-render scripts). The issue seems therefore to be in the order when pre-render scripts are ran.

I also got this to function with a minimal example using type: website as opposed to type: book, so the problems seems to lie in how books are constructed.

The discussion in #3879 shows that pre-render scripts used to be ran after Quarto has checked the project files against the specifications in _quarto.yml, but that this problem should be fixed now.

Since it works with websites but not books, I presume it is a bug of some sort?

Steps to reproduce

First, create a project folder structure like the example:

├── docs
│   ├── a.html
│   ├── b.html
│   └── index.html
├── index.qmd
├── post.R
├── pre.R
├── _quarto.yml
└── src
    ├── a
    │   └── a.qmd
    └── b
        └── b.qmd

Second, modify _quarto.yml to contain:

project:
    type: book
    output-dir: docs
    pre-render: pre.R
    post-render: post.R

book:
    title: "Testing"
    chapters:
        - index.qmd
        - a.qmd
        - b.qmd

format: html

Third, create the pre.R script:

# Save original file paths
orig_files <-
  fs::dir_ls("src", type = "file", recurse = TRUE)
write.csv(orig_files, "orig_files.csv")

# Move files into top-level
sapply(orig_files, fs::file_move, new_path = ".")

Fourth, create the post.R script:

# Find .qmd files to move back into original paths
moved_files <-
  fs::dir_ls(type = "file") |>
  grep("^(?!index\\.qmd$).+\\.qmd$", x = _, perl = TRUE, value = TRUE)
# The regex finds all .qmd files that are not named "index.qmd"

# Move files into original paths
orig_paths <- read.csv("orig_files.csv")[,1]
lapply(
  moved_files,
  function(file) {
    fs::file_move(
      file,
      grep(
        paste0("*", file),
        orig_paths,
        value = TRUE
      ))
  })

Finally, run quarto render from the top-level directory.

Expected behavior

The pre-render script is ran first, moving all files inside /src (recursively) into the top-level directory (and adding the csv file, which can be ignored for the purposes of this bug report):

├── docs
├── index.qmd
├── post.R
├── pre.R
├── orig_files.csv
├── a.qmd
├── b.qmd
├── _quarto.yml
└── src
    ├── a
    └── b

Quarto then renders the output into docs:

├── docs
│   ├── a.html
│   ├── b.html
│   └── index.html
├── index.qmd
├── post.R
├── pre.R
├── orig_files.csv
├── a.qmd
├── b.qmd
├── _quarto.yml
└── src
    ├── a
    └── b

Finally, the post-render script returns the files to their original directories:

├── docs
│   ├── a.html
│   ├── b.html
│   └── index.html
├── index.qmd
├── post.R
├── pre.R
├── orig_files.csv
├── _quarto.yml
└── src
    ├── a
    │   └── a.qmd
    └── b
        └── b.qmd

Actual behavior

ERROR: Book chapter 'a.qmd' not found

Stack trace:
    at throwInputNotFound (file:///home/osaal/opt/quarto-1.6.39/bin/quarto.js:99398:19)
    at findInputs (file:///home/osaal/opt/quarto-1.6.39/bin/quarto.js:99429:17)
    at async findChapters (file:///home/osaal/opt/quarto-1.6.39/bin/quarto.js:99441:13)
    at async bookRenderItems (file:///home/osaal/opt/quarto-1.6.39/bin/quarto.js:99444:5)
    at async Object.bookProjectConfig [as config] (file:///home/osaal/opt/quarto-1.6.39/bin/quarto.js:99349:25)
    at async projectContext (file:///home/osaal/opt/quarto-1.6.39/bin/quarto.js:81285:38)
    at async render (file:///home/osaal/opt/quarto-1.6.39/bin/quarto.js:90666:19)
    at async Command.actionHandler (file:///home/osaal/opt/quarto-1.6.39/bin/quarto.js:90843:32)
    at async Command.execute (file:///home/osaal/opt/quarto-1.6.39/bin/quarto.js:8111:13)
    at async Command.parseCommand (file:///home/osaal/opt/quarto-1.6.39/bin/quarto.js:8001:20)

Your environment

• OS: Linux (Fedora 40)
• R version 4.4.2 (2024-10-31) -- "Pile of Leaves"
• Quarto version 1.6.39
• No IDE, reproduced through Bash shell and Quarto CLI

Quarto check output

Quarto 1.6.39
[✓] Checking environment information...
      Quarto cache location: /home/osaal/.cache/quarto
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.4.0: OK
      Dart Sass version 1.70.0: OK
      Deno version 1.46.3: OK
      Typst version 0.11.0: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.6.39
      Path: /home/osaal/opt/quarto-1.6.39/bin

[✓] Checking tools....................OK
      Chromium: 869685
      TinyTeX: (not installed)

[✓] Checking LaTeX....................OK
      Using: Installation From Path
      Path: /usr/bin
      Version: 2023

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.10.14
      Path: /home/osaal/.pyenv/versions/3.10.14/bin/python3
      Jupyter: (None)

      Jupyter is not available in this Python installation.
      Install with python3 -m pip install jupyter

[✓] Checking R installation...........OK
      Version: 4.4.2
      Path: /usr/lib64/R
      LibPaths:
        - /home/osaal/R/x86_64-redhat-linux-gnu-library/4.4
        - /usr/lib64/R/library
        - /usr/share/R/library
      knitr: 1.48
      rmarkdown: 2.28

[✓] Checking Knitr engine render......OK

Reproduce the failure with the supplied _quarto.yml, pre.R, and post.R files, then trace the bookRenderItems, bookProjectConfig, and projectContext paths named in the stack trace. Compare the book flow with the working website case; the bug is fixed when the pre-render script runs before the book chapter existence check.