dapr / dapr/components-contrib
Standardize error messages in Dapr
- Dominant language
- Go
- Stars
- 602
- Forks
- 580
- Avg merge
- 4d 9h
- Merged PRs (30d)
- 6
Description
## Describe the proposal
For different building blocks the errors during init phase are surfaced in different ways. The following were the error logs when tested with component files with no required metadata for proper initialization of the component.
> Note: The line `error processing component, daprd process will exit gracefully` is a log line that is there for all init failures across building blocks.
#### State Store
```
## CosmosDB
WARN[0000] error initializing state store statestore (state.azure.cosmosdb/v1): url is required app_id=test-error instance=Mukundans-MacBook-Pro.local scope=dapr.runtime type=log ver=1.8.0
WARN[0000] error processing component, daprd process will exit gracefully app_id=test-error instance=Mukundans-MacBook-Pro.local scope=dapr.runtime type=log ver=1.8.0
```
```
## Cassandra
WARN[0000] error initializing state store statestore (state.cassandra/v1): missing or empty hosts field from metadata app_id=test-error instance=Mukundans-MacBook-Pro.local scope=dapr.runtime type=log ver=1.8.0
WARN[0000] error processing component, daprd process will exit gracefully app_id=test-error instance=Mukundans-MacBook-Pro.local scope=dapr.runtime type=log ver=1.8.0
```
> Note: For most of the building blocks the part `error initializing state store statestore (state.cassandra/v1):` differs by building block and the phrase after `:` is the one that is returned from the component code in components-contrib.
#### Pubsub
```
## Hazelcast
WARN[0000] error initializing pub sub pubsub.hazelcast/v1: hazelcast error: missing hazelcast servers app_id=test-error instance=Mukundans-MacBook-Pro.local scope=dapr.runtime type=log ver=1.8.0
WARN[0000] error processing component, daprd process will exit gracefully app_id=test-error instance=Mukundans-MacBook-Pro.local scope=dapr.runtime type=log ver=1.8.0
```
```
## Pulsar
WARN[0000] error initializing pub sub pubsub.pulsar/v1: pulsar error: missing pulsar host app_id=test-error instance=Mukundans-MacBook-Pro.local scope=dapr.runtime type=log ver=1.8.0
WARN[0000] error processing component, daprd process will exit gracefully app_id=test-error instance=Mukundans-MacBook-Pro.local scope=dapr.runtime type=log ver=1.8.0
```
#### Bindings
```
## AZ Blobstorage binding
WARN[0005] error processing component, daprd process will exit gracefully app_id=test-error instance=Mukundans-MacBook-Pro.local scope=dapr.runtime type=log ver=1.8.0
```
> Note: For all bindings errors, the log similar to `error initializing state store statestore (state.cassandra/v1): missing or empty hosts field from metadata` is missing. Only the generic component processing error is seen.
#### Nameresolution
```
## Consul
ERRO[0000] failed to initialize name resolution resolver consul: failed to register consul service: Put "http://127.0.0.1:8500/v1/agent/service/register": dial tcp 127.0.0.1:8500: connect: connection refused app_id=test-error instance=Mukundans-MacBook-Pro.local scope=dapr.runtime type=log ver=1.8.0
WARN[0000] failed to init name resolution: failed to register consul service: Put "http://127.0.0.1:8500/v1/agent/service/register": dial tcp 127.0.0.1:8500: connect: connection refused app_id=test-error instance=Mukundans-MacBook-Pro.local scope=dapr.runtime type=log ver=1.8.0
```
> Note: For Consul, its a bit different when required config parameters is not present or the service is not running for the init code to connect, we see a duplication of log messages, one at ERROR level and one at WARNING level.
These are the error messages that we see in the different components across different building blocks in components-contrib during the init phase.
For Publish errors, sometimes there are cases where we return a formatted [error](https://github.com/dapr/components-contrib/blob/master/pubsub/redis/redis.go#L166) in the component code and in some cases it is just returned [directly](https://github.com/dapr/components-contrib/blob/master/pubsub/jetstream/jetstream.go#L91) to be processed by runtime code.
For Statestore, sometimes there sepcific error wrap scenarios as seen [here](https://github.com/dapr/components-contrib/blob/master/state/errors.go), but for most cases, the errors are formatted as wanted by the component and sometimes the errors are returned directly as received from the client.
In some cases what we are seeing is that:
1. There incomplete standardization wrt logging and error messages during Init and shutdown.
1. There is no standardization across all components/building blocks on what the error message should contain (except the statestore Etag and BulkDelete are formatted consistently)
1. For built in retries, for some components, there are warning messages printed on a retry and for some that is not the case. And also on various components the log levels for these messages are completely different.
The issue here is further complicated when in the runtime, when gRPC API is used, the error is formatted and returned to the user in adifferent form whereas when HTTP is used, it is translated to HTTP status codes based responses with the error code (strings like `ERR_CONFIGURATION_STORE_NOT_CONFIGURED`) and message forming the part of the HTTP body.
The proposal in this issue is to make the flow of returning errors consistent across all components with slight differences with respect to the different building blocks.
Similar to [Heroku](https://devcenter.heroku.com/articles/error-codes), and similar to the partial solution that we already have for error codes, can we have a complete set of error codes, short description and hooks to create full fledged error responses based on the different lifecycle/API methods that we have for each building block?
For example each building block contains an `Init()` and `Close()` lifecycle methods (mostly all), and some other methods that are called during the init lifecycle is `Read()`, `Subscribe()` etc. which then spins up a go routine to handle input bindings or pubsub subscriptions.
For these lifecycle methods, we need to clearly emit metric and log when there are errors during the init phase and return clear formatted errors when response is sent back to the application.
For other methods where the API calls are user initiated, and the user waits for a response, clear formatted error messages would be better.
Similar to the how Etag Errors defined as seen below:
```go
// ETagError is a custom error type for etag exceptions.
type ETagError struct {
err error
kind ETagErrorKind
}
```
We can define one for all error scenarieos like:
```go
type DaprErrorCode string
ErrInitFailure DaprErrorCode = "D001"
ErrClose DaprErrorCode = "D100"
ErrPublishFailure DaprErrorCode = "D201"
// ...
type DaprError struct {
err error
code DaprErrorCode
shortDescription string
}
```
Where `code` is specific for certain error scenarios and is internal to Dapr, this will help with debug and support. The `shortDescription` can be a small description on what the error is, for example `ERR_CONFIGURATION_STORE_NOT_CONFIGURED`. And the `err` can be what we get back from a component. This struct will implement the `Error` interface and provide consistent formatting for error messages.
Additionally, we can define utility functions either based on each API/lifecycle method, for providing generic error codes and descriptions where the user would have to just call that method with the actual error they have to be wrapped into the DaprError format.
Contributor guide
Research direction
Start by comparing the linked pubsub/redis/redis.go and pubsub/jetstream/jetstream.go implementations with state/errors.go. Then trace how Init(), Close(), Read(), and Subscribe() errors move through the runtime's gRPC and HTTP paths. Done requires an agreed error-code and formatting model applied consistently across building blocks, lifecycle logging and metrics, and API responses.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend-api-design, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100