[FR] Allow to modified Meta in Lua to be write back in Markdown output with `+yaml_metadata_block`

Currently the below is not possible because in Quarto we do take over +yaml_metadata_block by recreating in a simpler manner leading to this limitation. We just do get the YAML header block in output to preprend in output
https://github.com/quarto-dev/quarto-cli/blob/295111c10fe3339125b8d928a4630a3b1085744e/src/command/render/output.ts#L142-L146

https://github.com/quarto-dev/quarto-cli/blob/295111c10fe3339125b8d928a4630a3b1085744e/src/command/render/output.ts#L149-L180

We remove it from Pandoc render
https://github.com/quarto-dev/quarto-cli/blob/295111c10fe3339125b8d928a4630a3b1085744e/src/command/render/pandoc.ts#L840-L843

So nothing modified in Meta inside Lua processing can be added into the output.

We need to modify the whole logic to be able to allow this.

In the first place, I believe we did this overwrite of feature from Pandoc to avoid having all Quarto's internal added to YAML meta showing in output.

For reference, history behind this at

• https://github.com/quarto-dev/quarto-cli/issues/4210

Discussed in https://github.com/quarto-dev/quarto-cli/discussions/10668

^{Originally posted by qiushiyan August 31, 2024}

Description

I am creating a lua filter in Quarto 1.5 to extract all headings from the source document, and want to add a headings field to the output gfm document.

Here is my lua filter and qmd

-- Function to create a slug from a string
local function slugify(s)
    return s:lower()
        :gsub("[^%w%s-]", "") -- remove non-word chars
        :gsub("%s+", "-")     -- replace spaces with hyphens
        :gsub("^-+", "")      -- trim starting hyphens
        :gsub("-+$", "")      -- trim ending hyphens
end

-- Table to store extracted headings
local headings = {}

-- Function to process headers
function Header(el)
    if el.level == 2 or el.level == 3 then
        local title = pandoc.utils.stringify(el.content)
        table.insert(headings, {
            depth = el.level,
            title = title,
            slug = slugify(title)
        })
    end
    return el
end

function Meta(meta)
    quarto.log.output("headings: " .. pandoc.utils.stringify(headings))
    meta["headings"] = headings
    return meta
end

---
format:
  gfm:
    variant: +yaml_metadata_block
    filters:
      - filters/headings.lua
---

# Main Title

## First Section

Some content here.

### Subsection A

More content.

## Second Section

Even more content.

### Subsection B

Final content.

During render, the filter is getting headings correctly and prints

(headings: 2first-sectionFirst Section3subsection-aSubsection A2second-sectionSecond Section3subsection-bSubsection B)

But no headings field is added to the output.

Start by reading src/command/render/output.ts around lines 142-180 and src/command/render/pandoc.ts around lines 840-843, then review issue 4210 and discussion 10668 for the existing design context. Done means Lua changes to Meta can be written into Markdown output with +yaml_metadata_block without exposing Quarto's internal metadata.