making commons data structures / tools available to external agents

The flow of data in commons

commons takes in:

• Data sources (DBI connections, tables, pins)
  • Sometimes with a data dictionary attached
• Semantic layer
  • Sometimes R code in a .R file, with sufficient docs to create an ellmer::tool
  • Sometimes built from elements of the data dictionary
• Context layer
  • Literally whatever free text

All of the above gets chunked up, with subsets included in the system prompt and tools:

• search_pool(): discover entries in the semantic layer
• call_measure() / call_metric(): call entries in the semantic layer
• describe_table(): HEAD on the table / relevant entries from the data dict
• search_context(): search inside of the semantic layer and context layer
• run_sql(): query the DB with free-handed SQL
• run_r(): (sandboxed) code execution, with run_sql() outputs serialized into the session

Interoperability with coding agents / commons

Some (notably, Canvas and Posit Assistant) MCP consumers have a code execution tool already.

The story here depends on whether we want to serve the end user or serve the data scientist that makes the agent.

Some options:

• MCP, with all tools
  • Maybe with run_r() optional
  • What to do about the system prompt? Technically compatible with MCP, but not first-class in mcptools (currently)
  • Easily deployable
  • "Hot" in enterprises
• Are data-dict.yaml and an R file are interoperable enough?
  • data-dict.yaml itself isn't actually interoperable if it has definitions (elements of the semantic layer)--in that case, you need commons or data-dict export (rust crate)
  • Maybe with a skill that shows agents how to interact with the files?
  • What about the connection to the data sources themselves?
• "Headless mode", maybe all in one tool
  • Analogous to $chat()
  • Maybe those who are more excited by this is developers
  • Could also be MCP
  • Subagent?
  • MCP tools can actually report progress, which could be use to stream responses
• Should Posit Assistant / Canvas be able to "consume" an ellmer chat?
  • MCP sorts of things, plus rich integration with ellmer/shinychat's Content types
• ACP?
• "Just run it from R"
  • Maybe recommend mcp-repl to do so (for non-Canvas and Posit Assistant)
• Plugin:
  • Is there a plugin-like thing for a data source?
  • Should commons agents be available as plugins?
• In Canvas, is there a first-class way to "import" a commons agent?
  • Is the commons runtime a "standard" that Canvas implements itself?
• What if Posit Assistant / Canvas supported some way to write plugins in R?
  • Some native ellmer/Chatlas integration (supporting streaming)

Start with the issue's data-flow list and compare the proposed MCP, headless-mode, file-based, plugin, and R-integration paths, including data-dict.yaml, R files, and the named tools. Before implementation, define which consumer and interoperability path is in scope; done requires a decided integration target and its behavior for tools, prompts, data sources, and streaming.