huggingface / huggingface/evaluate

HuggingFace evaluate-cli create command results Error: "datasets[0]" must be a string

Open
#514 2 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.5k
Forks
341
PR merge metrics
No merged PRs in 30d

Description

I follow the official documentation of `evaluate 0.4.0` to **freshly** create a custom metric in an empty folder with the following command:

evaluate-cli create "My Metric" --module_type "metric"

The command returns the following error:

Error: "datasets[0]" must be a string

The command creates these files but I am unsure whether the files are intact:

app.py my_metric.py README.md requirements.txt tests.py

How to resolve this error?

using `evaluate version 0.4.0`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.