[dotnet-ai] RAG pipeline
- Dominant language
- C#
- Stars
- 5.4k
- Forks
- 415
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 84
Description
## Parent Issue
Part of the `dotnet-ai` plugin proposal: #225
## Summary
Add the `rag-pipeline` skill to the `dotnet-ai` plugin.
This skill provides a self-contained guide for building end-to-end retrieval-augmented generation (RAG) pipelines in .NET. It composes four foundation skills into a coherent workflow: ingest documents → generate embeddings → store in vectors → retrieve and generate grounded answers.
## What This Skill Covers
A RAG pipeline has six stages, all covered:
| Stage | What Happens | Underlying Skill |
|---|---|---|
| 1. Document Ingestion | Load and chunk source documents | `data-ingestion-pipeline` |
| 2. Embedding Generation | Convert chunks to vector embeddings | `meai-embeddings` |
| 3. Vector Storage | Store embeddings with metadata in a vector store | `vector-data-search` |
| 4. Retrieval | Query the vector store with the user's question | `vector-data-search` |
| 5. Prompt Assembly | Combine retrieved context with the user's question | (this skill) |
| 6. Grounded Generation | Generate an answer grounded in retrieved context | `meai-chat-integration` |
**Additional coverage:**
- Source attribution — include provenance (document name, page, chunk ID) in the response
- Out-of-scope handling — detect when retrieved context doesn't contain the answer and respond honestly
- Relevance filtering — similarity threshold to avoid injecting irrelevant chunks
- Chunk overlap — overlapping windows during ingestion to preserve context across boundaries
## What This Skill Does NOT Cover
- Classical ML on structured data (use `mlnet`)
- Agentic workflows beyond simple RAG (use `agentic-workflow`)
- Building Copilot extensions (use `copilot-sdk-integration`)
## Files
```
plugins/dotnet-ai/skills/rag-pipeline/SKILL.md
tests/dotnet-ai/rag-pipeline/eval.yaml
```
## Eval Scenarios
1. **Build end-to-end RAG pipeline** — prompt asks to build a system where users can ask questions about internal documentation; expects all 6 RAG stages, source attribution, and out-of-scope handling
## Key Packages
- `Microsoft.Extensions.AI` (chat + embeddings)
- `Microsoft.Extensions.VectorData` (vector storage)
- `Microsoft.Extensions.DataIngestion` (document loading + chunking)
- Provider packages for LLM and embedding model
## Dependencies
Requires the plugin scaffold (sub-issue #1) to be merged first. Ideally the foundation skills (`meai-chat-integration`, `meai-embeddings`, `vector-data-search`, `data-ingestion-pipeline`) are merged first since this skill composes them and references them by name.
## Acceptance Criteria
- [ ] All 6 RAG stages are covered with code examples
- [ ] Source attribution guidance is present
- [ ] Out-of-scope handling (honest "I don't know") is documented
- [ ] Relevance threshold filtering is covered
- [ ] Skill explicitly references the 4 foundation skills it composes
- [ ] eval.yaml validates the end-to-end scenario
Contributor guide
Assessment
This issue has not been assessed yet.