avelino / avelino/dotagent

Add plugin: `sink-s3` (and S3-compatible endpoints)

Open
#21 0 comments 0 reactions 0 assignees View on GitHub
area: plugins type: feature
Dominant language
Rust
Stars
3
Forks
1
Avg merge
4h 50m
Merged PRs (30d)
1

Description

`sink-file` writes agent output to local disk. For agents that produce reports meant to be archived, shared with a team, or fed into downstream pipelines, "local disk" is rarely the right destination — you want S3 (AWS), GCS (GCP), or an S3-compatible bucket (R2, MinIO, Backblaze B2).

There is no sink plugin that targets object storage.

## Proposal

A new sink plugin `sink-s3` (which also handles S3-compatible endpoints).

Manifest usage:

```toml
[[on_success]]
plugin = "sink-s3"
bucket = "my-reports"
key = "reports/{agent}/{date}/{schedule}.txt" # template
endpoint = "https://r2.cloudflarestorage.com" # optional, defaults to AWS
region = "auto"
credentials = "env" # env | profile | iam-role
```

Templates available in `key`: `{agent}`, `{schedule}`, `{date}`, `{datetime}`, `{run_id}`.

For GCS, a sibling plugin `sink-gcs` may make sense or this plugin could grow a `flavor = "gcs"` field — implementer's call.

## Acceptance criteria

- [ ] New plugin crate under `plugins/sink-s3/`.
- [ ] Speaks the standard plugin protocol.
- [ ] Works against real AWS S3 and at least one S3-compatible service (smoke-tested with MinIO in CI is fine).
- [ ] Credentials sourced from env (`AWS_ACCESS_KEY_ID` etc.) or AWS shared config; never hardcoded in the manifest.
- [ ] Key template interpolation works.
- [ ] On upload failure, the plugin exits non-zero with a clear error; daemon logs it but does not crash.
- [ ] Doc page under `docs/plugins/sink-s3.md`.

## Where to start

- `plugins/sink-file/` and `plugins/sink-roam/` — convention.
- [`docs/reference/plugin-protocol.md`](docs/reference/plugin-protocol.md).
- `aws-sdk-s3` is the canonical crate but is heavy; evaluate `rust-s3` or `object_store` as lighter options.

## Non-goals

- Multipart uploads for large files. Agent output is rarely >100MB; single-PUT is fine for v1.
- Lifecycle management (expiry, glacier tier). Configure that at the bucket level.

Contributor guide

Open the contributing guide

Research direction

Read plugins/sink-file/ and plugins/sink-roam/ for plugin conventions, then consult docs/reference/plugin-protocol.md. Decide between the listed Rust storage libraries, implement the sink-s3 crate with credential handling, endpoint support, and key templates, and add docs/plugins/sink-s3.md. Done means the standard protocol works with AWS S3 and an S3-compatible service, failures are clear and non-zero, and the daemon remains running.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, rust
Domain
cloud
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.