ConduitIO / ConduitIO/conduit-commons
paramgen: add support for custom field (un)marshalling
- Dominant language
- Go
- Stars
- 3
- Forks
- 3
- Avg merge
- 1d 24m
- Merged PRs (30d)
- 3
Description
### Feature description
**The problem**: Here's an example from a connector middleware struct:
```go
type SourceWithSchemaExtraction struct {
SchemaType schema.Type `json:"sdk.schema.extract.type" validate:"inclusion=avro" default:"avro"`
}
```
where `schema.Type` is an alias of [schema.Type in conduit-commons](https://github.com/ConduitIO/conduit-commons/blob/main/schema/schema.go#L30).
When parsing `SourceWithSchemaExtraction` with `sdk.Util.ParseConfig`, we get an error saying that there's a type mismatch, because the default value is `"avro"` (a string), and the target value is an `int`. The reason is that in [`Config.Validate()`](https://github.com/ConduitIO/conduit-commons/blob/main/config/config.go#L69) we validate the types.
**Possible solution**:
1. Remove type validation in `Config.Validate()` altogether. In the Connector SDK, it's called only when parsing a config, and after that we decode the map into a struct, where the types are validated too.
2. Use one of `mapstructure`'s hooks ([link](https://github.com/mitchellh/mapstructure/pull/222)), so that developers can implement a `MarshalText`/`UnmarshalText` method and customize parsing.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing sdk.Util.ParseConfig through Config.Validate and the subsequent mapstructure decoding path. Compare removing validation with adding mapstructure hooks for MarshalText/UnmarshalText, using the custom schema.Type example as the acceptance case. Done means custom field unmarshalling handles string defaults without breaking existing config validation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100