microsoft/m365-copilot-eval

M365 Copilot Evaluations CLI

Stars: 17
Forks: 1
Open beginner issues: 0
Indexed issues: 6

Dominant language: Python
License: No license data
Last GitHub push: Jul 31, 2026
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: Code of conduct
Beginner labels: No beginner labels indexed
PR merge metrics: No merged PRs in 30d

Beginner friendly Everything

Open All

6 open issues indexed

Support asserting a specific SharePoint knowledge source in RetrievalResult Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

microsoft/m365-copilot-eval#37 · Opened Aug 11, 2026
Does a usage-based billing spending policy alone unblock the A2A 403 "NoPolicy: AI credits access is not configured for this user"? Open

Difficulty 3/5 1-2 days Newbie friendliness 48/100

microsoft/m365-copilot-eval#32 · 1 comment · Opened Jul 6, 2026
Evaluators crash (uncaught) instead of failing the item gracefully on empty responses / unparseable judge output Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

microsoft/m365-copilot-eval#30 · Opened Jun 28, 2026
Surface HTTP 429 throttling as a structured signal (distinct error code + run summary + Retry-After) so wrapping tools can warn / back off Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

microsoft/m365-copilot-eval#29 · 1 comment · Opened Jun 28, 2026
Feature: evaluate a pre-recorded response (offline/replay) without re-invoking the agent Open

Difficulty 4/5 3-5 days Newbie friendliness 68/100

microsoft/m365-copilot-eval#27 · Opened Jun 26, 2026
Support for DA that invokes 3P MCP tools authenticated via OAuthPluginVault Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

microsoft/m365-copilot-eval#16 · 2 comments · Opened Jun 10, 2026

A short digest of beginner-friendly GitHub issues.