[Feature] Support Kafka SCRAM authentication in Kafka Client monitoring
- Dominant language
- Java
- Stars
- 7.4k
- Forks
- 1.3k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 32
Description
## Feature Request
Add SCRAM authentication support to the Kafka Client monitoring template and its Java-side AdminClient configuration.
## Is your feature request related to a problem? Please describe
The current Kafka Client template cannot monitor Kafka clusters that require SASL/SCRAM authentication.
Updating `app-kafka_client.yml` alone is insufficient. The YAML template can add authentication fields to the page and pass their values into the kclient protocol configuration, but the Java code that creates Kafka `AdminClient` must also put those values into Kafka `Properties`.
At present, the official implementation only passes the host, port, and `monitorInternalTopic`. The official documentation does not provide SASL parameters, so the built-in template cannot connect to SCRAM-protected Kafka clusters.
Documentation: https://hertzbeat.apache.org/zh-cn/docs/help/kafka_client/
## Describe the solution you would like
Please support SCRAM authentication end to end:
- Add optional authentication fields to `app-kafka_client.yml`, including username, password, SASL mechanism, and security protocol.
- Pass these values through the kclient protocol configuration.
- When creating Kafka `AdminClient`, populate the corresponding Kafka properties, such as `security.protocol`, `sasl.mechanism`, and `sasl.jaas.config`.
- Support at least `SCRAM-SHA-256` and `SCRAM-SHA-512`.
- Keep the existing unauthenticated behavior backward compatible when authentication is not configured.
- Avoid exposing the password in logs or diagnostic output.
- Update the Kafka Client documentation with a SCRAM configuration example.
## Describe alternatives you have considered
Maintaining a locally modified YAML template and Java implementation is possible, but it diverges from the official template and must be maintained across upgrades. YAML-only customization cannot solve the issue because the final Kafka `Properties` are created in Java.
## Additional context
A typical Kafka client configuration requires values equivalent to:
```properties
security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-256
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="..." password="...";
```
`SASL_SSL` should also be usable when TLS is required.
Contributor guide
Research direction
Start by reading app-kafka_client.yml and tracing how its values enter the kclient protocol configuration. Then inspect the Java code that creates Kafka AdminClient and the linked Kafka Client documentation. Done means optional SCRAM-SHA-256/512 settings work for SASL_PLAINTEXT and SASL_SSL, unauthenticated behavior remains compatible, passwords are not logged, and the documentation includes an example.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, kafka
- Domain
- observability-sre
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100