airbytehq / airbytehq/airbyte

[source-facebook-marketing] -> [BigQuery] MERGE query for streams made with custom insights configuration are only loading 1 record per account id

Đang mở
#38,088 9 bình luận 0 reaction 0 người được giao Xem trên GitHub
area/connectors autoteam community connectors/source/facebook-marketing team/critical-connectors team/use type/bug
Ngôn ngữ chính
Python
Star
22.1k
Fork
5.3k
Chỉ số merge pull request
Chỉ số pull request đang chờ

Mô tả

### Connector Name

source-facebookmarketing

### Connector Version

2.1.7

### What step the error happened?

Other

### Relevant information

I have a custom insights stream configured like this:
Fields
- account_id
- account_name
- campaign_id
- campaign_name
- spend
- impressions
- clicks
- date_start
- date_stop

Level
- campaign

The connection insists that `ad_id` is a primary key for this custom insights configuration
![image](https://github.com/airbytehq/airbyte/assets/107578051/ad33b5ae-6ba2-4b45-ad49-f4b8ecf86a1a)

Naturally in the destination table this field value is NULL
![image](https://github.com/airbytehq/airbyte/assets/107578051/ed6373b7-1a9a-41b5-aee5-e61576c7e21a)

I've noticed that our data doesn't match up with the Excel extractions from the platform, the gap is very large.

It is clear from the MERGE query, that the culprit is the row numbering which occurs in a window that includes `date`, `account_id`, `ad_id` (NULL).

This causes the merge query to load only the 1st record extracted for each ad account, consequently including only 1 campaign.
![image](https://github.com/airbytehq/airbyte/assets/107578051/a521e711-d2e5-4640-9c39-bff3d523d0ed)

We've verified that the destination table includes only 1 campaign per account, which does not reflect the marketing activity of these ad accounts.

I have tried refreshing source schema, but the `ad_id` persists in the custom insights report.

I haven't dug into the source code, so I can't tell why this happens, though I assume the custom insights `level` parameter should be enough to hint which fields can be used as primary key, and this is not respected.

### Relevant log output

_No response_

### Contribute

- [ ] Yes, I want to contribute

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.