vectordotdev / vectordotdev/vector
AWS authentication for the kafka source and sink
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 22.6k
- Forks
- 2.3k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 146
Description
I did some light reading regarding Amazon's MSK service authentication strategy:
https://docs.aws.amazon.com/msk/latest/developerguide/security_iam_service-with-iam.html
It appears we'll need to support the basic AWS authentication strategy for both the kafka source and sink. We accomplished this in the elasticsearch sink by adding an auth strategy:
[sinks.es]
type = "elasticsearch"
auth.strategy = "aws"
I would like to do the same here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read Amazon's MSK IAM authentication documentation and inspect how the Elasticsearch sink implements its AWS auth strategy. Trace the Kafka source and sink authentication entry points, then verify that both accept the corresponding auth configuration and can authenticate against MSK.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, kafka, rust
- Domain
- authentication, stream-processing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100