kestra-io / kestra-io/plugin-compress

Add Directory Compression Support to ArchiveCompress

Open
#129 0 comments 0 reactions 1 assignee Claimed by @Malaydewangan09 View on GitHub
area/plugin
Dominant language
Java
Stars
3
Forks
8
Avg merge
3d 8h
Merged PRs (30d)
5

Description

### Feature description

### Problem

`ArchiveCompress` currently only supports compressing individual files with explicit mapping. It **cannot compress directories recursively**, making it impractical for directories with 100+ files.

**Current:** ✅ Works with file maps only
```yaml
from:
file1.txt: "{{ outputs.task1.uri }}"
file2.txt: "{{ outputs.task2.uri }}"
```

**Needed:** ❌ Doesn't work with directories
```yaml
from: "kestra:///namespace/path/to/directory" # or local filesystem path
```

### Use Cases

- Archiving entire directories for backup
- Compressing large datasets with 100+ files
- Dynamic file sets where names/counts are unknown
- Compressing all outputs from previous workflow tasks

### Proposed Solution

Add `fromDirectory` property (or enhance `from` to auto-detect directories):

```yaml
tasks:
- id: archive_compress
type: io.kestra.plugin.compress.ArchiveCompress
fromDirectory: "kestra:///namespace/path/to/directory"
algorithm: ZIP
```

### Technical Requirements

1. Auto-detect if input is a directory vs file
2. Recursively walk directory structure
3. Preserve relative paths in archive
4. Support namespace storage URIs and local filesystem paths
5. Maintain backward compatibility with existing map-based usage

**Note:** `ArchiveDecompress` already handles directory structures correctly when extracting.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.