Azure / Azure/azure-sdk-for-python
metric name does not match
Offen
#45,905
3 Kommentare
1 Reaktion
4 zugewiesene Personen
Beansprucht von @slister1001
Auf GitHub ansehen
customer-reported
Evaluation
needs-team-attention
question
Service Attention
- Vorherrschende Sprache
- Python
- Sterne
- 5.6k
- Forks
- 3.4k
- Ø Merge
- 2 T. 2 Std.
- Gemergte PRs (30 T.)
- 213
Beschreibung
- **Package Name**: azure-ai-evaluation
- **Package Version**: 1.16.2
- **Operating System**:linux
- **Python Version**: 3.11
**Describe the bug**
The metrics are missing after evaluation. After investigating, it seems that:
for example with self_harm evaluation:
SDK sends ["name": "self_harm"] in the request payload (snake_case)
API (2025-11-15-preview) returns ["metric": "SelfHarm"] in the response (PascalCase)
SDK in _parse_eval_result() compares "SelfHarm" == "self_harm" → False → result discarded
Because of this, all metrics are discarded in the evaluation process.
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.