GoogleCloudPlatform / GoogleCloudPlatform/gsutil

Allow the destination of 'cp', 'rsync', and similar commands to be specified as an option

Open
#1,307 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
918
Forks
335
PR merge metrics
No merged PRs in 30d

Description

## The Problem

When using `gsutil cp` and `gsutil rsync` in combination with shell pipes, it's impossible to use tools like `xargs --no-run-if-empty` of `find -exec ... \+` because `gsutil` requires that the destination be the last positional argument.

If the destination could be specified by a command-line argument, that would allow for more effective use of `xargs` and `find`.

Here are some examples of things that don't work the way I'd hope due to the destination being a required positional argument:

* '-m' is rendered useless because `xargs -I{}` limits the number of args to 1 per command, so gsutil is invoked for each file found.
```
find /path -type f -mmin 15 -print0 | \
xargs -0 --no-run-if-empty -I {} gsutil -m cp {} gs://destination
```

* '-m' is rendered useless because `find ... -exec ... \;` calls gsutil once for each file found.
```
find /path -type f -mmin 15 -exec gsutil -m cp {} gs://destination \;
```

* This fails because `find ... -exec ... \+` is incapable of putting the replaced text anywhere except the end of the command.
```
find /path -type f -mmin 15 -exec gsutil -m cp {} gs://destination \+
```

## The Request
Allow the destination to be specified by an argument. For example ` -t` or `--target` or `--destination`.

By doing that, these commands would work and be able to take advantage of `gsutil`'s -m flag.
```
find /path -type f -mmin 15 -print0 | \
xargs -0 --no-run-if-empty gsutil -m cp --destination gs://destination
```
```
find /path -type f -mmin 15 -exec gsutil -m cp --destination gs://destination {} \+
```

Contributor guide

Open the contributing guide

Research direction

Start at the gsutil cp and gsutil rsync command entry points and inspect how positional source and destination arguments are parsed. Define the destination option behavior for both commands, including batching with xargs and find -exec; done means the requested examples work while preserving existing positional usage.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.