vectordotdev / vectordotdev/vector
AWS S3 sink healthchecks require permissive ACLs
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 22.6k
- Forks
- 2.3k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 146
Description
For the aws_s3 sink, writes to the S3 bucket only require file-level s3:PutObject permissions, however healthchecks are done via HeadBucket which requires the bucket-level s3:ListBucket permissions.
While I'm sure this is fine for some use cases, it does mean giving writers access to view the entire bucket's contents, even if they are only writing to a specific prefix, which seems too permissive given it's just for healthchecks.
The value of a healthcheck here is also questionable given that it doesn't simulate the permissions/API call required to actually store a log file.
I would suggest using the same approach as AWS ALB access logging of writing a test file to
{{ bucket_name }}/{{ key_prefix }}/AccessLogTestFile
Or alternatively giving an option to disable/ignore healthchecks.
In my use-case where there are multiple AWS accounts writing to a single logging bucket, giving s3:ListBucket would mean allowing any account to list log files written by other accounts (as well as other hosts in its own account), whereas for example, the preferred ACL for my use-case would be:
EventsPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: ...
PolicyDocument:
Version: 2012-10-17
Statement:
- Sid: ChildAccountDeliveryForAcc90000000000
Effect: Allow
Principal:
AWS: 90000000000
Action: s3:PutObject
Resource: arn:aws:s3:::logging-bucket/AppLogs/90000000000/*
- Sid: ChildAccountDeliveryForAcc91111111111
Effect: Allow
Principal:
AWS: 91111111111
Action: s3:PutObject
Resource: arn:aws:s3:::logging-bucket/AppLogs/91111111111/*
which can be achieved with the key_prefix setting, assuming heathchecks aren't required/enforced.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the aws_s3 sink healthcheck implementation and inspect how HeadBucket and key_prefix are used. Compare the healthcheck permissions with the permissions required for writing a log file, then determine whether a write-based check or an option to disable healthchecks best fits the stated use case. Done means the sink no longer requires unnecessary bucket listing access while preserving clear healthcheck behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, rust
- Domain
- cloud
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100