Azure / Azure/azure-sdk-for-python
metric name does not match
Open
customer-reported
Evaluation
needs-team-attention
question
Service Attention
- Dominant language
- Python
- Stars
- 5.6k
- Forks
- 3.4k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 193
Description
- **Package Name**: azure-ai-evaluation
- **Package Version**: 1.16.2
- **Operating System**:linux
- **Python Version**: 3.11
**Describe the bug**
The metrics are missing after evaluation. After investigating, it seems that:
for example with self_harm evaluation:
SDK sends ["name": "self_harm"] in the request payload (snake_case)
API (2025-11-15-preview) returns ["metric": "SelfHarm"] in the response (PascalCase)
SDK in _parse_eval_result() compares "SelfHarm" == "self_harm" → False → result discarded
Because of this, all metrics are discarded in the evaluation process.
Contributor guide
Assessment
This issue has not been assessed yet.