AlibabaResearch / AlibabaResearch/DAMO-ConvAI
Evaluation Method for BIRD Dataset [Enhancement]
Abierto
- Lenguaje dominante
- Python
- Estrellas
- 1.6k
- Forks
- 250
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Descripción
Hello,
I encountered an improvement opportunity during the evaluation process for the BIRD dataset. The prediction below is marked as incorrect by the evaluation method, but the only difference is the order of the elements.
The evaluation method uses strict equality. This occurs in the file [bird/llm/src/evaluation.py](https://github.com/AlibabaResearch/DAMO-ConvAI/blob/main/bird/llm/src/evaluation.py) on line 26.
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Evaluación
Este issue todavía no se ha evaluado.