posit-dev / posit-dev/positron

Agents cannot execute or observe Quarto documents in inline execution mode (epic: positron.quarto context API)

Open
#14,733 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area: assistant area: notebooks-quarto epic info needed
Dominant language
TypeScript
Stars
4.3k
Forks
184
Avg merge
1d 11h
Merged PRs (30d)
206

Description

Problem

When a Quarto document runs in inline execution mode (positron.quarto.inlineOutput.enabled), its kernel session is started as a notebook-mode session with notebookUri pointing at the .qmd file (quartoKernelManager.ts, deliberate). When the .qmd editor is focused, that session becomes the foreground session, and positron.runtime.getForegroundSession() reports sessionMode === 'notebook'.

Posit Assistant (and any other agent surface) is therefore steered into notebook tools and away from executeCode -- but no notebook editor instance backs the session, so every notebook tool fails, and there is no tool-reachable way to execute code in the document's live kernel or observe its outputs at any level:

Agent action Result
executeCode Error: "active session is a notebook session ... use notebookEdit/notebookRunCells"
notebookRead/notebookEdit/notebookRunCells (no path) Error: "No notebook is open in the editor."
same tools with path: report.qmd Error: "not a Jupyter notebook (.ipynb). Use the text tools"
read/edit on the .qmd Works, but blind: cannot run chunks, cannot see chunk outputs or execution state
shell (Rscript -e, quarto render) Fresh out-of-process interpreter: no access to the live session's state; prompt discourages it

The error messages form a loop (executeCode says "use notebook tools"; notebook tools say "no notebook is open" / "use text tools"; text tools cannot run anything). The model typically burns several failing tool calls, then edits the document blind or falls back to out-of-session shell execution.

Repro
  1. Set positron.quarto.inlineOutput.enabled: true.
  2. Open a report.qmd with an {r} chunk (e.g. mean(mtcars$mpg)), run the chunk inline so the kernel starts.
  3. With the .qmd editor focused, ask the assistant: "Run the first chunk and tell me the result."
  4. Observe the failure loop in the tool calls.
Why this is not #14250

#14250 is the same symptom family for .ipynb in the legacy Jupyter editor, where the workaround is "reopen in the Positron Notebook Editor". A .qmd can never be opened in a notebook editor: the session is notebook-mode by design and the document is plain text by design, so no in-session workaround exists. Tightening the notebooks API around PNE instances (the likely #14250 direction) cannot resolve this case.

The gap, precisely

Every piece of state an agent needs already exists in workbench services under src/vs/workbench/contrib/positronQuarto/ -- parsed chunk structure (IQuartoDocumentModelService), execution state (IQuartoExecutionManager), outputs with content-hash staleness (IQuartoOutputManager + IQuartoOutputCacheService) -- but none of it crosses the extension-host boundary. Meanwhile the session metadata does cross the boundary and actively misidentifies the document class ("notebook") with no discriminator other than sniffing the notebookUri file extension.

The only ext-host-facing surfaces today are positron.runtime.executeInlineCell(uri, ranges) (returns void -- caller observes nothing) and session-targeted positron.runtime.evaluateCode (text-level results only, silent side-band evaluation).

Plan (recommended design from the investigation)

Full analysis with candidate designs A-D lives in the investigation branch (investigate/quarto-inline-agent-context, PROPOSAL-quarto-inline-agent-context.md). Recommendation: Design B (first-class positron.quarto context API in core), with C.1 (assistant-side session classification + prompt fix) shipped first as an immediate mitigation. C.1 is a strict subset of B's assistant half.

Design A (overloading positron.notebooks to resolve Quarto documents) was rejected: it deepens the notebook/Quarto conflation that caused this bug and pulls against #14250. Design D (console-mode Quarto sessions) was rejected: session mode is load-bearing for lifecycle and variables-pane integration.

API sketch (Design B)
positron.quarto.getContext(): Thenable<QuartoDocumentContext | undefined>
// { uri, executionMode, primaryLanguage, kernelState, sessionId?,
//   chunks: [{ id, label?, language, startLine, endLine, content,
//              executionStatus, hasOutput, outputStale }] }
positron.quarto.getChunkOutputs(uri, chunkId): Thenable<NotebookCellOutput[]>
positron.quarto.executeChunks(uri, chunkIds): Thenable<ChunkExecutionResult[]>

Anchored on the text editor + document model (mirroring how the notebook context anchors on the notebook editor instance). Decisions already made:

  • Namespace: positron.quarto (rename only if a second inline-document format materializes).
  • .Rmd in scope (core services already accept it via isQuartoOrRmdFile).
  • Flag off (positron.quarto.inlineOutput.enabled: false): getContext() resolves read-only (chunk structure, no kernel/execution state); executeChunks errors clearly.
  • Visual editor mode: deferred; v1 anchors on the active text editor.
  • Staleness/read path anchors on the contentHash-keyed output cache (chunk IDs rotate on edit; the live output map has no id-remap).
  • executeChunks re-resolves chunk addresses server-side (label where present, else content hash / containing line) and errors on ambiguity.
  • Outputs carrying webviewMetadata (interactive/HTML view zones) get a defined downgrade (text fallback or explicit marker).

Tasks

  • C.1 (posit-dev/assistant): classify quarto-inline sessions (by notebookName extension), add a quarto-session.md prompt include replacing the misfiring notebook-session.md, and make the executeCode refusal message document-class-aware (keep the refusal: falling through would route code to a console, not the document's kernel).
  • Core (this repo): positron.quarto API -- ext-host/main-thread pair over the four existing positronQuarto services, DTOs, positron.d.ts, tests.
  • Assistant adoption: quartoRead (allow) + quartoRunChunks (ask) tools over the new API; edits stay with text tools; update vendored typings.
  • Follow-up: core-side document-type discriminator in session metadata, so agent surfaces stop sniffing file extensions.
  • Related UX (separate issue): #14732 -- stale output view zones are not marked/cleared when chunk source is edited.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the existing services under src/vs/workbench/contrib/positronQuarto/ and the ext-host-facing positron.runtime APIs described in the issue. Review the proposed positron.quarto context, output, and chunk-execution surfaces, then trace the related positron.d.ts and test areas. Done requires an ext-host/main-thread API with DTOs and tests that exposes Quarto context, outputs, and execution without routing through notebook tools.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
api, developer-experience
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.