TAR Workflow for sealed-copy
- Dominant language
- Rust
- Stars
- 9
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
How great would it be if godi could handle tar files natively ?! Being dependent on files is one thing, but streaming everything, is another.
Whatever is done with tars, all features currently available should natively be supported. A TAR is as good as a directory source, or destination.
## Phase 1: Produce TARs
The invocation could look as follows
``` bash
godi sealed-copy --verify A/ B/ -- foo.tar - /Volumes/D/
```
The invocation above hashes and seals all contents of A/ and B/ to a file called foo.tar. If it would already exist as directory, it is interpreted as directory, of course. If it is a file, this is an error.
The destination specified as `-` signals to write to stdout as TAR formatted stream. This will force all logging activity to STDERR
Last but not least, the final destination is a directory.
Shall everything succeed, the result is as follows:
- **foo.tar**
- Contains A/... and B/..., as well as a standard seal as last file. It uses relative paths only.
- **-**
- Is the streamed equivalent of foo.tar
- **/Volumes/D
- Equals foo.tar if it would have been extracted there.
Verification will only be done on `/Volumes/D` in Phase 1, and additionally on the TAR file in Phase 2.
### Caveats
- TARs, especially those to STDOUT, cannot be verified afterwards. In that case, just skip them.
- Streamed TARs cannot be rolled back, obviously
## Phase 2: Consume Tars
This would look like this. Note the similar syntax for the input specification
``` bash
godi sealed-copy --verify - foo.tar S/ -- - bar.tar D/
```
The line above will read all streamed files from STDIN as TAR, from foo.tar as TAR if and only if it is a file, and from the directory S/.
The merged content of all these sources is streamed to STDOUT in TAR format, to bar.tar (only if it is no directory) in TAR format, and to D/ as files. All destinations will of course have their own seal file respectively.
### Caveats
- `seal` doesn't really make sense with tars as input, but might be allowed for completeness
- To stream in TARS, we need to keep this information in memory somehow. We might just create disk-based memory maps in some temp location, to support TARs of any size. They must be removed after consumption. This feature would certainly require an abstracted ReadChannelController.
## Workflow Examples
- **Seal Existing TARs**
``` bash
godi sealed-copy my.tar -- my.sealed.tar /Volumes/Backup/my
godi sealed-copy - -- - /Volumes/Backup/my < my.tar > my.sealed.tar
```
```
* Similarly, tar can be used to resolve and follow symbolic links in a directory, feeding the result to godi. That way, one can overcome a godi limitation, which will either ignore symlinks, or reproduce them as such.
```
- **Cryptographically Strong and Insured Stream**
- This one might be useful for fully automated digital deliveries.
``` bash
godi sealed-copy Delivery/ -- - | gpg --encrypt --recipient ClientID | deliver-to ClientID
```
Contributor guide
Research direction
Start at the `godi sealed-copy` command and trace how directory sources, destinations, verification, logging, and streaming are currently handled. Compare those paths with the Phase 1 and Phase 2 TAR workflows, including the proposed `ReadChannelController` abstraction. Done means TAR files and streams support the existing sealed-copy features, with the stated verification and rollback limitations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- cli
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100