huggingface / huggingface/evaluate
Too Long Package Import
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 341
- PR merge metrics
- No merged PRs in 30d
Description
Package `evaluate` takes too much time to import while it is not so heavy as TensorFlow or Torch.
```shell
$ time python -c 'import torch; print(torch.ones(3))' >> /dev/null
python -c 'import torch; print(torch.ones(3))' 0,84s user 0,06s system 95% cpu 0,942 total
$ time python -c 'import evaluate'
python -c 'import evaluate' 2,23s user 0,19s system 98% cpu 2,453 total
```
It takes 2.5 seconds against 0.9 seconds for PyTorch! Moreover CLI takes the same time and fails on startup.
```shell
$ time evaluate-cli
Traceback (most recent call last):
File "/usr/bin/evaluate-cli", line 8, in
sys.exit(main())
File "/usr/lib/python3.10/site-packages/evaluate/commands/evaluate_cli.py", line 72, in main
if args["module_type"] not in ["metric", "comparison", "measurement"]:
KeyError: 'module_type'
evaluate-cli 2,31s user 0,17s system 99% cpu 2,490 total
```
Contributor guide
Assessment
This issue has not been assessed yet.