GoogleCloudPlatform / GoogleCloudPlatform/gsutil
Compress downloads
- Dominant language
- Python
- Stars
- 918
- Forks
- 335
- PR merge metrics
- No merged PRs in 30d
Description
## Compression on downloads
It would be nice to be able to compress downloads (-z and -Z). This has its use for those who require/desire the local compression.
I imagine this might present issues with consistency between the local and cloud-bucket data, and those issues might need to be resolved (with naming and file tests, for instance), for best functionality. I'm new to gcloud services, and am unfamiliar with its workflow.
An example of this feature, while not necessarily a contemporary type of functionality for cp'ish commands, is:
gsutil cp -Z bucket/*.txt localdir/ # This creates a local .txt.gz file for each file in bucket/
I presently am using a script which runs through an ls -r, creates the directories, then cat's each file, piped into gzip -c9.
## Notes:
- The compression can be done client-side, if server-side gzip is an issue and provides no greater benefits.
- Other options, like -j for bzip2, might also be useful then, matching some 'tar' programs' options.
- cat -Z might also be good.
- By default, I think downloading a .gz should cause a recompression, to .gz.gz. A separate option, say, -d for now, would handle this issue. E.g: "-d When using -z/-Z, do not double-compress. By default, with -z/Z, .gz files will be compressed and renamed to .gz.gz . This option keeps them as .gz."
Contributor guide
Assessment
This issue has not been assessed yet.