microsoft/rag-experiment-accelerator

View on GitHub

The RAG Experiment Accelerator is a versatile tool designed to expedite and facilitate the process of conducting experiments and evaluations using Azure Cognitive Search and RAG pattern.

Stars: 311
Forks: 111
Open beginner issues: 0
Indexed issues: 98

Dominant language: Python
License: No license data
Last GitHub push: Apr 1, 2025
Latest indexed: Sep 19, 2026
Contributing guide: No contributing guide
Code of conduct: Code of conduct
Beginner labels: No beginner labels indexed
PR merge metrics: No merged PRs in 30d

Beginner friendly Everything

Open All

98 open issues indexed

azd up command causing the error "downloading bicep: Get "https://downloads.bicep.azure.com/v0.38.33/bicep-linux-x64": context canceled" Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

microsoft/rag-experiment-accelerator#868 · Opened Nov 16, 2025
KeyBasedAuthenticationNotPermitted Open

Difficulty 4/5 3-5 days Newbie friendliness 20/100

microsoft/rag-experiment-accelerator#866 · Opened Mar 4, 2025
Add Support for Custom Embeddings Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

microsoft/rag-experiment-accelerator#819 · Opened Nov 4, 2024
Include page number and file name as metadata for each chunk Open

microsoft/rag-experiment-accelerator#818 · 1 assignee · Opened Nov 4, 2024
Make RAGE work with CAIRA Open

microsoft/rag-experiment-accelerator#817 · 1 assignee · Opened Nov 4, 2024
Implement semantic chunking within RAG EXP ACC Open

microsoft/rag-experiment-accelerator#816 · 1 comment · 1 assignee · Opened Nov 4, 2024
Working with local models Open

microsoft/rag-experiment-accelerator#815 · 1 assignee · Opened Nov 4, 2024
Indexing step not working properly when "use_checkpoints" is turned on Open

microsoft/rag-experiment-accelerator#783 · 1 assignee · Opened Oct 14, 2024
rag-experiment-accelerator package installation fails on PromptFlow due to dependency conflicts Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

microsoft/rag-experiment-accelerator#755 · Opened Sep 24, 2024
[peer-review-pal] Repository Review for Commit #290bd0ecdde69903f6792d281f4afc6e6604f09c Open

Difficulty 5/5 Over a week Newbie friendliness 15/100

microsoft/rag-experiment-accelerator#754 · Opened Sep 23, 2024
Add source to pdf doc metadata Open

Good have hack

microsoft/rag-experiment-accelerator#735 · 1 assignee · Opened Sep 18, 2024
pip install custom_environment/rag_experiment_accelerator-0.9-py3-none-any.whl fails Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

microsoft/rag-experiment-accelerator#734 · Opened Sep 18, 2024
[Refactoring] Pass config to `load_<format>_files()` methods Open

Good have hack

microsoft/rag-experiment-accelerator#732 · 1 assignee · Opened Sep 17, 2024
Test Extracting a table from a PDF Open

microsoft/rag-experiment-accelerator#729 · 1 comment · 1 assignee · Opened Sep 16, 2024
Evaluate Unstructured PDF Reader Library Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

microsoft/rag-experiment-accelerator#727 · Opened Sep 16, 2024
Read a PDF file and if it has images then it should be converted into embededdings and stored with AI Search Open

hack

microsoft/rag-experiment-accelerator#725 · 3 assignees · Opened Sep 16, 2024
Add promptflow-evals quality metrics as an alternative to ragas Open

enhancement

microsoft/rag-experiment-accelerator#707 · 1 comment · 1 assignee · Opened Sep 13, 2024
enable image and table data ingestion, indexing and search as part of Multi-modal RAG Open

Good have hack

microsoft/rag-experiment-accelerator#703 · 1 comment · 3 assignees · Opened Sep 9, 2024
add new Search Features like Filtering,(pre, post), generate dynamic rich metadata (like summary, title, keywords, entities etc) along with usage of Query Profiles Open

enhancement hack

Difficulty 5/5 Over a week Newbie friendliness 15/100

microsoft/rag-experiment-accelerator#685 · 1 comment · Opened Aug 29, 2024
index_name is cut off to 128 characters Open

Must have

Difficulty 5/5 Over a week Newbie friendliness 25/100

microsoft/rag-experiment-accelerator#684 · Opened Aug 29, 2024
Integrate the best experiment (prompt Flow) with LLMOps for automated deployment Open

enhancement hack

Difficulty 5/5 Over a week Newbie friendliness 15/100

microsoft/rag-experiment-accelerator#669 · Opened Aug 14, 2024
Generate Prompt Flow inference pipeline based on the best experiment configuration Open

enhancement hack

Difficulty 5/5 Over a week Newbie friendliness 15/100

microsoft/rag-experiment-accelerator#668 · Opened Aug 14, 2024
Extend metric reporting such that confidence intervals are also computed and displayed. Confidence interval computation should leverage recent frameworks like prediction-powered inference to combine AI-based metrics (like LLM-based metrics) with human evaluations. Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

microsoft/rag-experiment-accelerator#597 · 1 reaction · Opened Jun 13, 2024
Add groundedness evaluation metric Open

Difficulty 5/5 Over a week Newbie friendliness 30/100

microsoft/rag-experiment-accelerator#573 · Opened May 23, 2024
Documentation around why SpacyEvaluator is used Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

microsoft/rag-experiment-accelerator#551 · Opened May 13, 2024
Separate each search type into its own mlflow run to allow comparison using Azure ML / mlflow Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

microsoft/rag-experiment-accelerator#540 · Opened May 9, 2024
Custom PdfLoader implementation support Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

microsoft/rag-experiment-accelerator#539 · Opened May 7, 2024
Add RAFT capabilities, fine-tuning and inference Open

enhancement hack Sprint 2

microsoft/rag-experiment-accelerator#527 · 1 assignee · Opened May 2, 2024
Scenario based sample evaluation Open

enhancement hack Sprint 2

microsoft/rag-experiment-accelerator#526 · 2 assignees · Opened May 2, 2024
Scenario based sample configuration Open

enhancement hack Sprint 2

Difficulty 5/5 Over a week Newbie friendliness 15/100

microsoft/rag-experiment-accelerator#525 · Opened May 2, 2024
Modularize REA repository Open

enhancement hack Sprint 2

Difficulty 5/5 Over a week Newbie friendliness 15/100

microsoft/rag-experiment-accelerator#524 · Opened May 2, 2024
Consolidate and refactor configuration based on RAG Triad Open

Sprint 2

microsoft/rag-experiment-accelerator#523 · 1 comment · 1 assignee · Opened May 2, 2024
Reconstruct CI/CD pipeline for different scenarios including local and Azure ML execution Open

Must have Sprint 2

Difficulty 5/5 Over a week Newbie friendliness 25/100

microsoft/rag-experiment-accelerator#522 · 1 comment · Opened May 2, 2024
Bring better Q&A generation capabilities Open

Good have

Difficulty 5/5 Over a week Newbie friendliness 15/100

microsoft/rag-experiment-accelerator#505 · Opened Apr 24, 2024
Bicep deployment template lacks an embedding model Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

microsoft/rag-experiment-accelerator#492 · Opened Apr 18, 2024
Implementing Checkpoints for Azure ML Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 30/100

microsoft/rag-experiment-accelerator#491 · Opened Apr 18, 2024
Adding Checkpoint usage to additional steps (already implemented for 01_index)" Open

enhancement Must have

microsoft/rag-experiment-accelerator#490 · 1 assignee · Opened Apr 18, 2024
02_qa_generation fails with exception Open

bug

Difficulty 2/5 1-3 hours Newbie friendliness 42/100

microsoft/rag-experiment-accelerator#485 · 1 comment · Opened Apr 16, 2024
Part 2: Bicep script to manage access to resources in isolated network Open

Must have

Difficulty 4/5 3-5 days Newbie friendliness 35/100

microsoft/rag-experiment-accelerator#466 · Opened Apr 11, 2024
PRs from Forks won't have Access to Secrets Open

github_actions Must have

Difficulty 4/5 3-5 days Newbie friendliness 35/100

microsoft/rag-experiment-accelerator#452 · 1 comment · 1 reaction · Opened Apr 2, 2024
AzureML pipeline: Add step for qna generation Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

microsoft/rag-experiment-accelerator#435 · Opened Mar 27, 2024
Introduce opencensus logging Open

Good have

microsoft/rag-experiment-accelerator#431 · 1 comment · 1 assignee · Opened Mar 26, 2024
AzureML pipeline: Update bicep configuration Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 38/100

microsoft/rag-experiment-accelerator#411 · Opened Mar 19, 2024
CI/CD support for PromptFlow Open

Difficulty 4/5 3-5 days Newbie friendliness 42/100

microsoft/rag-experiment-accelerator#409 · Opened Mar 18, 2024
CI/CD support for AzureML Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

microsoft/rag-experiment-accelerator#408 · Opened Mar 18, 2024
Consistent file naming Open

refactoring

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

microsoft/rag-experiment-accelerator#407 · Opened Mar 18, 2024
LanguageEvaluator class is unused Open

bug

Difficulty 5/5 Over a week Newbie friendliness 25/100

microsoft/rag-experiment-accelerator#405 · Opened Mar 18, 2024
PromptFlow pipeline runs on outdated, fixed version of the code Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

microsoft/rag-experiment-accelerator#404 · 1 comment · 1 reaction · Opened Mar 18, 2024
AzureML pipeline: Ability to enable and disable steps Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

microsoft/rag-experiment-accelerator#403 · Opened Mar 18, 2024
AzureML pipeline: Use Docker image for the environment Open

microsoft/rag-experiment-accelerator#402 · 1 assignee · Opened Mar 18, 2024

microsoft/rag-experiment-accelerator

Get new issues in your inbox