dapr / dapr/components-contrib
[PubSub Kafka] Some important error logs are misclassified as `debug` instead of `error`
- Dominant language
- Go
- Stars
- 602
- Forks
- 580
- Avg merge
- 4d 9h
- Merged PRs (30d)
- 6
Description
**runtime 1.12.2**
## Expected Behavior
Errors reported from the Sarama client should be raised with a `error` log level in Dapr (not `debug`)
## Actual Behavior
Errors are misclassified as `debug` but they should be `error` - need to ensure errors are classified correctly so they can be accurately reported in observability tools (i.e. datadog in my case)
## A few examples of the errors :
`level=debug msg="kafka: error while consuming ds-applicationdatabricks-created/0: read tcp 10.0.2.72:43382->10.0.0.132:9092: i/o timeout"`
`level=debug msg="kafka: error while consuming ds-applicationdatabricks-created/0: kafka: client has run out of available brokers to talk to: 3 errors occurred:
\t* dial tcp: lookup luma-kafka-kafka-2.luma-kafka-kafka-brokers.dotm-services.svc: i/o timeout
\t* dial tcp: lookup luma-kafka-kafka-1.luma-kafka-kafka-brokers.dotm-services.svc on 172.20.0.10:53: read udp 10.0.2.72:38664->172.20.0.10:53: read: connection refused
\t* write tcp 10.0.2.72:59042->172.20.14.72:9092: write: broken pipe"`
`level=debug msg="kafka: error while consuming ds-applicationrealm-user-created/0: kafka: client has run out of available brokers to talk to: 3 errors occurred:
\t* dial tcp: lookup luma-kafka-kafka-0.luma-kafka-kafka-brokers.dotm-services.svc on 172.20.0.10:53: read udp 10.0.2.72:39002->172.20.0.10:53: read: connection refused
\t* dial tcp: lookup luma-kafka-kafka-1.luma-kafka-kafka-brokers.dotm-services.svc on 172.20.0.10:53: no such host
\t* dial tcp 172.20.14.72:9092: connect: connection refused
"`
`level=debug msg="kafka: error while consuming ds-applicationrealm-created/0: kafka: client has run out of available brokers to talk to: dial tcp 172.20.14.72:9092: connect: connection refused" app_id=authz component="general-purpose-pubsub (pubsub.kafka/v1)"`
Contributor guide
Research direction
Start at the Go Kafka PubSub component and trace where Sarama consumer errors are logged, using the example messages in the issue as the expected cases. Done means the listed consumer, broker, and network errors are emitted at error level rather than debug and are accurately visible to observability tools.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kafka
- Domain
- stream-processing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100