feldera / feldera/feldera

Validate connector config/properties for tables

Open
#3,968 6 comments 0 reactions 1 assignee Claimed by @ryzhyk View on GitHub
connectors high priority user-reported
Dominant language
Rust
Stars
2.1k
Forks
154
Avg merge
2d 19h
Merged PRs (30d)
136

Description

To avoid confusion and "stupid" miss config it would be practical with an error message if you try to add an no valid/real attribute to the `connectors` section in table properties.

I made a mistake by putting the `append_only` attribute under `connectors` and not at top-level. I got no error message and program run fine, but of course the table was not append only. I then discovered that you can put arbitrary attribute under connectors without any errors, e.g `abc`.

Maybe add an check that attributes under connectors is an allowed/real one? I see that there is value validation for valid attributes, for example `mode` for delta input connector have value validation, checking that is is "snapshot", "follow", "snapshot_follow" or "cdc".

The following example vil run without any errors and problems. But table will not be an append only table.
```
CREATE TABLE my_table (
...
) WITH (
'connectors' = '[{
"abc"= "123",
'append_only': 'true'
...
}]'
)
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.