potential breaking API changes from alpacal_eval?
- Lenguaje dominante
- Python
- Estrellas
- 395
- Forks
- 105
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Descripción
I'm using `'preference_version': 'alpaca_eval==0.6.6'` and the current alpaca_eval_v2 in this library will throw me a key error because apparently have been change `price_per_example` to `'preference_price_per_example'`, in which the following line is affected: https://github.com/allenai/olmes/blob/38af8b61741b01faeb60e90899addd7e6fa503a0/oe_eval/tasks/oe_eval_tasks/alpaca_eval.py#L171
As well as `time_per_example`, looks like there is only `preference_price_per_example` in an annotation. Everything has been prepended with "preference". So should we pin the alpaca_eval version in this library?
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Evaluación
Este issue todavía no se ha evaluado.