dagucloud / dagucloud/dagu

Add Secure Secret Management Support (Vault, AWS, GCP)

Open
#798 2 comments 2 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
4k
Forks
332
Avg merge
19h 9m
Merged PRs (30d)
137

Description

**Overview**
We want to enhance Dagu to securely manage secrets by retrieving them from Vault, AWS Secrets Manager, or GCP Secret Manager. Since these devices can be physically stolen, our goal is to ensure secrets are never stored in plain text on the device and are only loaded at execution time with minimal exposure.

**Proposal**
1. **Secret Providers Configuration**
- Extend DSL to include a `secretProviders` section for Vault, AWS, and GCP.
- Example snippet:
```yaml
secretProviders:
vault:
address: "${VAULT_ADDR}"
token: "${VAULT_TOKEN}"
aws:
region: "${AWS_REGION}"
accessKeyId: "${AWS_ACCESS_KEY_ID}"
secretAccessKey: "${AWS_SECRET_ACCESS_KEY}"
gcp:
credentialsJSONPath: "${GCP_CREDENTIALS_JSON}"
```

2. **DAG File `secrets:` Section**
- Introduce a `secrets:` block in each DAG to declare which provider to use, the secret’s path/ARN, and an internal reference name.
- On execution, Dagu retrieves the requested secrets and injects them as temporary environment variables for the relevant steps.
- Example snippet:
```yaml
secrets:
- name: DB_PASSWORD
source: vault
path: "secret/data/db_credentials"
key: "password"
- name: AWS_KEY
source: aws
arn: "arn:aws:secretsmanager:us-east-1:123456789012:secret:myToken-xxxx"

steps:
- name: migrate
command: ./migrate.sh --db-pass "$DB_PASSWORD"
- name: upload
command: aws s3 cp data.out s3://my-bucket/ --access-key "$AWS_KEY"
depends: migrate
```

3. **Security Considerations**
- **No Plain Text Logging:** Ensure logs/UI never expose secret values, potentially masking them.

Any feedbacks on the design, or thoughts are very welcomed.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.