GoogleCloudPlatform / GoogleCloudPlatform/gsutil
Proposal to switch from crcmod to crc32c
Open
- Dominant language
- Python
- Stars
- 918
- Forks
- 335
- PR merge metrics
- No merged PRs in 30d
Description
crcmod uses a simple lookup table (compiled from C).
[crc32c](https://github.com/ICRAR/crc32c) uses hardware instructions for Intel and ARMv8 CPUs, with software fallback, and is significantly faster. It also has binary builds available for most platforms, which makes it much easier to install (you could vendor it in your third_party directory even).
`time gsutil hash -c (400 mb file)`
* crcmod: 7.103 seconds
* crc32c: 3.048 seconds
Contributor guide
Assessment
This issue has not been assessed yet.