airbytehq / airbytehq/airbyte

source-hubspot: `marketing_emails.testing.isAbVariation` and `owners.teams[].primary` are returned by the API but absent from the schemas, so V2 destinations drop them silently

Open
#82,771 2 comments 0 reactions 0 assignees View on GitHub
autoteam community connectors/source/hubspot hyd-prove hyd-review team/use
Dominant language
Python
Stars
22.1k
Forks
5.3k
PR merge metrics
PR metrics pending

Description

### Connector Name

source-hubspot

### Connector Version

6.7.0

### What step the error happened?

During the sync

### Relevant information

Two properties are returned by the HubSpot API but not declared in the stream schemas:

| Stream | Property | Endpoint |
|---|---|---|
| `marketing_emails` | `testing.isAbVariation` | `/marketing/v3/emails` |
| `owners` | `teams[].primary` | Owners API |

This used to be harmless. The older S3/GCS Avro writers collected undeclared properties into an `_airbyte_additional_properties` catch-all, so the values still reached the warehouse.

That is no longer true on V2 destinations. Per the [S3 migration guide](https://docs.airbyte.com/integrations/destinations/s3-migrations), which made the file destinations "isomorphic to our V2 certified database destinations", "undocumented fields in object with schemas are silently dropped" and "`_airbyte_additional_properties` is dropped entirely". Nothing signals it either — "dropped fields will not appear in `_airbyte_meta.changes[]`".

So on any V2 destination both values disappear without a trace, and the only clue is a column that quietly stops being populated.

Observed on a production connection:

- `testing.isAbVariation` was present on 787 of 6 745 `marketing_emails` records before the destination upgrade, and on 0 after.
- `teams[].primary` sits in the `owners` catch-all; `teams[]` declares only `id`, `name` and `membership`.

The same migration guide notes that a schemaless object "usually indicates an upstream source is failing to report its schema properly", which is exactly the situation here: the fix belongs in the source schema.

### Relevant log output

_No response_

### Contribute

- [x] Yes, I want to contribute

Fixed by #82769.

---
**Internal Tracking:** https://github.com/airbytehq/oncall/issues/13166

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.