airbytehq / airbytehq/airbyte

source-goldcast: `event_members.props` enumerates a fixed list of workspace-defined registration fields, so V2 destinations silently drop every other one

Aperta
#82,772 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub
autoteam community connectors/source/goldcast hyd-review team/use
Lingua principale
Python
Stelle
22.1k
Fork
5.3k
Metriche di merge delle PR
Metriche PR in attesa

Descrizione

### Connector Name

source-goldcast

### Connector Version

0.2.25

### What step the error happened?

During the sync

### Relevant information

`event_members.props` holds Goldcast **registration form fields**, which each workspace defines for itself. The schema nevertheless enumerates a fixed list of eleven of them:

```
city, solutions, tag_source, utm_source, utm_medium, utm_campaign,
utm_content, tag_country, tag_form_type, revenue_type, contact_job_title
```

Any registration field a workspace defines outside that list is not declared, and on V2 destinations undeclared properties are gone. Per the [S3 migration guide](https://docs.airbyte.com/integrations/destinations/s3-migrations), which made the file destinations "isomorphic to our V2 certified database destinations", "undocumented fields in object with schemas are silently dropped" and "`_airbyte_additional_properties` is dropped entirely" — with no signal at all, since "dropped fields will not appear in `_airbyte_meta.changes[]`".

The older S3/GCS Avro writers hid this by gathering the extra keys into the catch-all. Now they simply vanish.

Because the field names are workspace-specific, the enumerated list can only ever be right for the workspaces whose names happen to be in it. Declaring them one at a time also means a new PR against this repo every time any user adds a registration field — which does not scale and leaves everyone else silently losing data in the meantime.

The same migration guide points at the general remedy: "objects without schemas are serialized into a JSON string", and "objects with null or empty properties are treated as schemaless objects". Making `props` schemaless would preserve **every** key for **every** workspace, dynamically, with no further schema PRs.

That is a breaking change for anyone reading `props.` as a nested column today, so it warrants a major version bump and a migration note.

### Relevant log output

_No response_

### Contribute

- [x] Yes, I want to contribute

Addressed by #82770.

---
**Internal Tracking:** airbytehq/oncall#13167

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.