GoogleCloudPlatform / GoogleCloudPlatform/gsutil

Compress downloads

Open
#438 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
918
Forks
335
PR merge metrics
No merged PRs in 30d

Description

## Compression on downloads
It would be nice to be able to compress downloads (-z and -Z). This has its use for those who require/desire the local compression.

I imagine this might present issues with consistency between the local and cloud-bucket data, and those issues might need to be resolved (with naming and file tests, for instance), for best functionality. I'm new to gcloud services, and am unfamiliar with its workflow.

An example of this feature, while not necessarily a contemporary type of functionality for cp'ish commands, is:
gsutil cp -Z bucket/*.txt localdir/ # This creates a local .txt.gz file for each file in bucket/

I presently am using a script which runs through an ls -r, creates the directories, then cat's each file, piped into gzip -c9.

## Notes:
- The compression can be done client-side, if server-side gzip is an issue and provides no greater benefits.
- Other options, like -j for bzip2, might also be useful then, matching some 'tar' programs' options.
- cat -Z might also be good.
- By default, I think downloading a .gz should cause a recompression, to .gz.gz. A separate option, say, -d for now, would handle this issue. E.g: "-d When using -z/-Z, do not double-compress. By default, with -z/Z, .gz files will be compressed and renamed to .gz.gz . This option keeps them as .gz."

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.