huggingface / huggingface/evaluate
CI parity test is broken due to FileNotFoundError
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 341
- PR merge metrics
- No merged PRs in 30d
Description
CI parity test is broken due to FileNotFoundError: https://github.com/huggingface/evaluate/actions/runs/10827111652/job/30039577026?pr=631
```
FAILED tests/test_trainer_evaluator_parity.py::TestEvaluatorTrainerParity::test_token_classification_parity - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpg8va97zxevaluator_trainer_parity_test/tokenclassification_conll2003_transformers/eval_results.json'
```
```
_________ TestEvaluatorTrainerParity.test_token_classification_parity __________
[gw0] linux -- Python 3.8.18 /opt/hostedtoolcache/Python/3.8.18/x64/bin/python
self =
def test_token_classification_parity(self):
model_name = "hf-internal-testing/tiny-bert-for-token-classification"
n_samples = 500
subprocess.run(
"git sparse-checkout set examples/pytorch/token-classification",
shell=True,
cwd=os.path.join(self.dir_path, "transformers"),
)
subprocess.run(
f"python examples/pytorch/token-classification/run_ner.py"
f" --model_name_or_path {model_name}"
f" --dataset_name conll2003"
f" --do_eval"
f" --output_dir {os.path.join(self.dir_path, 'tokenclassification_conll2003_transformers')}"
f" --max_eval_samples {n_samples}",
shell=True,
cwd=os.path.join(self.dir_path, "transformers"),
)
> with open(
os.path.join(self.dir_path, "tokenclassification_conll2003_transformers", "eval_results.json"), "r"
) as f:
E FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpg8va97zxevaluator_trainer_parity_test/tokenclassification_conll2003_transformers/eval_results.json'
tests/test_trainer_evaluator_parity.py:293: FileNotFoundError
```
Contributor guide
Assessment
This issue has not been assessed yet.