ConduitIO / ConduitIO/conduit-connector-postgres

Feature: Handle Composite Types

Open
#275 3 comments 0 reactions 0 assignees View on GitHub
feature
Dominant language
Go
Stars
19
Forks
12
Avg merge
12h 20m
Merged PRs (30d)
5

Description

### Feature description

I have a project that I would like to use conduit for but it makes use of composite types e.g.

```
CREATE TYPE public.money_with_currency AS (currency_code char(3), amount numeric(20,8))
```

When I attempt to start to run a pipeline, I receive:

```
ERR node stopped error="node pg-to-log:pg stopped with error: source stream was stopped unexpectedly: error reading from source: read plugin error: failed to fetch next record: fetchers exited unexpectedly: fetcher for table \"payment_transactions\" exited: failed to fetch results: failed to extract schemas: failed to extract payload schema for table payment_transactions: field \"amount\" with OID 20272 cannot be resolved" component=lifecycle.Service node_id=pg-to-log:pg stack=[{"file":"/private/tmp/conduit-20250320-4805-o24r34/conduit-0.13.3/pkg/lifecycle/service.go","func":"github.com/conduitio/conduit/pkg/lifecycle.(*Service).runPipeline.func2","line":743},{"file":"/private/tmp/conduit-20250320-4805-o24r34/conduit-0.13.3/pkg/lifecycle/stream/source.go","func":"github.com/conduitio/conduit/pkg/lifecycle/stream.(*SourceNode).Run","line":146},{"file":"/private/tmp/conduit-20250320-4805-o24r34/conduit-0.13.3/pkg/lifecycle/stream/source.go","func":"github.com/conduitio/conduit/pkg/lifecycle/stream.(*SourceNode).Run.func1","line":90}]
```

Since it's possible to list the composite types that have been created detailed here: https://dba.stackexchange.com/questions/35497/display-user-defined-types-and-their-details

schema_name | obj_name | column_name | data_type | ordinal_position | is_required | description
-------------+---------------------+-------------+--------------+------------------+-------------+-------------
public | money_with_currency | currency | character(3) | 1 | f |
public | money_with_currency | amount | numeric | 2 | f |
public | typed_id | type | text | 1 | f |
public | typed_id | id | bigint | 2 | f |
public | typed_refid | type | text | 1 | f |
public | typed_refid | refid | text | 2 | f |

Is it possible to support composite types by either creating Avro records/maps or at the least serializing them to json or string?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.