E3SM-Project / E3SM-Project/simboard
[AI]: Add curated documentation retrieval for AI summary citations
- Dominant language
- Python
- Stars
- 1
- Forks
- 2
- Avg merge
- 24m
- Merged PRs (30d)
- 15
Description
## Objective
SimBoard AI summaries need stronger source grounding for explanatory context, caveats, and citations.
Phase 3 should add a small curated retrieval path, starting with the v3.LR.historical ensemble Confluence page:
https://e3sm.atlassian.net/wiki/spaces/CM/pages/4182474754/v3.LR.historical+ensemble
This is not a broad RAG platform, AI agent, MCP integration, or source-code indexing effort.
## Scope
Add backend retrieval for curated documentation context in the existing AI summary workflow.
Initial source:
- v3.LR.historical ensemble Confluence export
- Use for: ensemble purpose, science context, caveats, terminology, interpretation notes
- Do not use for: simulation metadata, run status, provenance, execution IDs, artifact paths, or diagnostics availability
SimBoard metadata remains the source of truth for simulation-specific facts.
## Implementation notes
- Export the Confluence page to stable Markdown.
- Store source metadata: ID, title, original URL, export date, owner, scope, source type, authority level.
- Add a small retrieval abstraction rather than hardcoded prompt injection.
- Pass normalized retrieved context into the AI summary service.
- Return citation metadata with AI summary responses.
- Log retrieved source IDs and trace IDs.
Suggested interface:
`retrieve_context(query, simulation_metadata, limit) -> list[RetrievedContext]`
Each result should include source ID, title, URL/path, heading, excerpt, rank if available, and export/version metadata.
## Acceptance criteria
- AI summaries can retrieve context from the curated Confluence export.
- Documentation-grounded claims include citations.
- Simulation-specific claims still come from SimBoard metadata.
- Missing or unsupported context is shown as a caveat or limitation.
- Retrieval failure does not break baseline summary generation.
- Retrieval is behind a backend abstraction that can later connect to a larger RAG or knowledge service.
- Tests cover retrieval success, empty results, citation formatting, fallback behavior, and unsupported-claim avoidance.
## Out of scope
- Full RAG system
- AI agent workflow
- MCP integration
- Full Confluence sync
- Source-code indexing
- Open-ended doc chat
- Web retrieval
- User-uploaded document ingestion
- Production vector database
## Related
- Phase 3 of SimBoard AI assistance prototype
- Related issue: #52
Contributor guide
Assessment
This issue has not been assessed yet.