cockroachdb / cockroachdb/cockroach
cdc: support human-readable parse rules (e.g. "1MiB") in kafka_sink_config integer fields
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
**Is your feature request related to a problem? Please describe.**
When configuring `kafka_sink_config`, integer fields such as `Flush.MaxBytes` require raw `int32` values. Attempting to use human-readable byte size strings (e.g. `"1MiB"`) results in an error:
```
root@localhost:26257/defaultdb> ALTER CHANGEFEED 1161831199211847681 SET kafka_sink_config = '{"Flush": {"Frequency": "100ms", "MaxBytes": "1MiB"}, "RequiredAcks": "ALL"}';
ERROR: failed to alter changefeed: failed to parse sink config; check kafka_sink_config option: json: cannot unmarshal string into Go struct field .Flush.MaxBytes of type int32
```
This is inconsistent with other fields like `Flush.Frequency`, which already accepts human-readable duration strings (e.g. `"100ms"`).
**Describe the solution you'd like**
Support human-readable byte size parse rules (e.g. `"1MiB"`, `"512KiB"`) for integer fields in `kafka_sink_config` that represent byte quantities, such as `Flush.MaxBytes`. This would make the configuration more ergonomic and consistent with the existing duration string support.
**Describe alternatives you've considered**
Users can currently provide raw integer values (e.g. `1048576` instead of `"1MiB"`), but this is less readable and user-friendly.
**Additional context**
Discovered during manual testing of backport PRs for the Kafka coalescing fix (#166302). See Slack discussion: https://cockroachlabs.slack.com/archives/C09DHJ4KL68/p1774634324367809?thread_ts=1774030024.172079&cid=C09DHJ4KL68
This does not need to be backported — forward-only improvement.
Jira issue: CRDB-62131
Contributor guide
Assessment
This issue has not been assessed yet.