Embeded chunks do not respect the "execute-dir: project" setting

Bug description

I have a quarto project, in which I have set the "execute-dir: project" in the yml file. This is because I have data files in one folder in the project dir, and quarto notebooks in a docs folder. When an individual notebook loads data files it works. However, if I embed a chunk from such a notebook into another, suddenly I get an error that the file cannot be found. Seems like rendering of notebooks to embed them does not respect the execture-dir: project setting.

Steps to reproduce

# create a quarto website project
tmp <- tempfile()
dir.create(tmp)
setwd(tmp)
system("quarto create project website .")

# create a file in a folder
dir.create("inner")
file.copy("about.qmd", "inner/file.qmd")

# write a code chunk in the inner file to open a file in base directory
saveRDS(rnorm(10), "data.rds")
CON <- file("inner/file.qmd", "a")
writeLines("```{r embedme}\nprint(readRDS('data.rds'))\n```", CON)
close(CON)

# ensure that the execture-dir is set to the project directory
writeLines("project:
  type: website
  execute-dir: project

website:
  title: 'blabla'
  navbar:
    left:
      - href: index.qmd
        text: Home
      - about.qmd

format:
  html:
    theme: cosmo
    css: styles.css
    toc: true", "_quarto.yml")

# write an index file that embeds the embedme chunk of the inner file
CON <- file("index.qmd", "a")
writeLines("{{< embed inner/file.qmd#embedme >}}", CON)
close(CON)

quarto::quarto_render()

#> [1/3] inner/file.qmd
#> 
#> 
#> processing file: file.qmd
#> 1/3          
#> 2/3 [embedme]
#> 3/3          
#> output file: file.knit.md
#> 
#> [2/3] index.qmd
#> Rendering qmd embeds
#> [1/1] inner/file.qmd
#> 
#> 
#> processing file: file.qmd
#> 1/3          
#> 2/3 [embedme]
#> 
#> Quitting from lines  at lines 9-10 [embedme] (file.qmd)
#> Error in `gzfile()`:
#> ! cannot open the connection
#> Backtrace:
#>  1. base::print(readRDS("data.rds"))
#>  2. base::readRDS("data.rds")
#>  3. base::gzfile(file, "rb")
#> Execution halted
#> ERROR: Rendering of qmd notebook produced an unexpected result
#> Error in `quarto::quarto_render()`:
#> ✖ Error running quarto cli.
#> Caused by error:
#> ! System command 'quarto' failed

^{Created on 2024-05-13 with reprex v2.1.0}

You can see from the output that the inner\file.qmd is rendered succesfully by itself. However, when it is rendered to create and embeded chunk, it fails. This is not due to using quarto::quarto_render() - the same happens with the CLI, but the reprex does not show the output...

Expected behavior

Rendering of notebooks via embedings (https://quarto.org/docs/authoring/notebook-embed.html#overview) should respect the _quarto.yml options

Actual behavior

it does not respect the execute-dir: project option

Your environment

• IDE: irrelevant, happens even from the terminal
• MacOS Sonoma 14.4

Quarto check output

Quarto 1.4.553
[✓] Checking versions of quarto binary dependencies...
Pandoc version 3.1.11: OK
Dart Sass version 1.69.5: OK
Deno version 1.37.2: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
Version: 1.4.553
Path: /Applications/quarto/bin

[✓] Checking tools....................OK
TinyTeX: (not installed)
Chromium: (not installed)

[✓] Checking LaTeX....................OK
Tex: (not detected)

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
Version: 3.12.1 (Conda)
Path: /opt/miniconda3/bin/python
Jupyter: 5.5.0
Kernels: ir, python3

[✓] Checking Jupyter engine render....OK

[✓] Checking R installation...........OK
Version: 4.3.3
Path: /opt/homebrew/Cellar/r/4.3.3/lib/R
LibPaths:
- /opt/homebrew/lib/R/4.3/site-library
- /opt/homebrew/Cellar/r/4.3.3/lib/R/library
knitr: 1.46
rmarkdown: 2.26

[✓] Checking Knitr engine render......OK