huggingface / huggingface/tokenizers
Rust Issue on Unix in 0.21.0 Version
- Dominant language
- Rust
- Stars
- 11k
- Forks
- 1.2k
- Avg merge
- 3d 8h
- Merged PRs (30d)
- 26
Description
Hi,
While installing the tokenizers library after 0.21.0 version release, the installation is failing in Unix. (Python 3.9.16)
It is looking for a Rust installation.
12:27:55 Looking in indexes: https://pypi.org/simple
12:27:55 Collecting tokenizers (from -r requirements.txt (line 1))
12:27:56 Downloading https:///pypi/pypi-/tokenizers-0.21.0.tar.gz (343 kB)
12:27:56 Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'error'
error: subprocess-exited-with-error
× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [6 lines of output]
Checking for Rust toolchain....
Cargo, the Rust package manager, is not installed or is not on PATH.
This package requires Rust and Cargo to compile extensions. Install it through
the system's package manager or via https://rustup.rs/
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
When I revert to 0.20.3 , it works fine.
Contributor guide
Assessment
This issue has not been assessed yet.