Add Secure Secret Management Support (Vault, AWS, GCP)
- Dominant language
- Go
- Stars
- 4k
- Forks
- 332
- Avg merge
- 19h 9m
- Merged PRs (30d)
- 137
Description
**Overview**
We want to enhance Dagu to securely manage secrets by retrieving them from Vault, AWS Secrets Manager, or GCP Secret Manager. Since these devices can be physically stolen, our goal is to ensure secrets are never stored in plain text on the device and are only loaded at execution time with minimal exposure.
**Proposal**
1. **Secret Providers Configuration**
- Extend DSL to include a `secretProviders` section for Vault, AWS, and GCP.
- Example snippet:
```yaml
secretProviders:
vault:
address: "${VAULT_ADDR}"
token: "${VAULT_TOKEN}"
aws:
region: "${AWS_REGION}"
accessKeyId: "${AWS_ACCESS_KEY_ID}"
secretAccessKey: "${AWS_SECRET_ACCESS_KEY}"
gcp:
credentialsJSONPath: "${GCP_CREDENTIALS_JSON}"
```
2. **DAG File `secrets:` Section**
- Introduce a `secrets:` block in each DAG to declare which provider to use, the secret’s path/ARN, and an internal reference name.
- On execution, Dagu retrieves the requested secrets and injects them as temporary environment variables for the relevant steps.
- Example snippet:
```yaml
secrets:
- name: DB_PASSWORD
source: vault
path: "secret/data/db_credentials"
key: "password"
- name: AWS_KEY
source: aws
arn: "arn:aws:secretsmanager:us-east-1:123456789012:secret:myToken-xxxx"
steps:
- name: migrate
command: ./migrate.sh --db-pass "$DB_PASSWORD"
- name: upload
command: aws s3 cp data.out s3://my-bucket/ --access-key "$AWS_KEY"
depends: migrate
```
3. **Security Considerations**
- **No Plain Text Logging:** Ensure logs/UI never expose secret values, potentially masking them.
Any feedbacks on the design, or thoughts are very welcomed.
Contributor guide
Assessment
This issue has not been assessed yet.