fivetran / fivetran/dbt_shopify

Error in models/shopify__transactions.sql Related to Calculated Exchange Rates for Refunds

Open
#69 6 comments 0 reactions 0 assignees View on GitHub
status:stale type:bug
Dominant language
No language data
Stars
82
Forks
47
Avg merge
5d 4h
Merged PRs (30d)
4

Description

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Describe the issue

The exchange rate related columns are calculated via this code:

```
select
*,
coalesce(cast(nullif({{ fivetran_utils.json_parse("receipt",["charges","data",0,"balance_transaction","exchange_rate"]) }}, '') as {{ dbt.type_numeric() }} ),1) as exchange_rate,
coalesce(cast(nullif({{ fivetran_utils.json_parse("receipt",["charges","data",0,"balance_transaction","exchange_rate"]) }}, '') as {{ dbt.type_numeric() }} ),1) * amount as currency_exchange_calculated_amount
from joined
```

However, the structure of the JSON object in the `receipt` column is different for refunds. This leads to the `exchange_rate` field always falling back to the coalesced value of 1 for refunds transactions. This is subtle for conversions close to 1.0 but become very problematic for conversions rates that are not close to 1.0. For example, CRC which has a rate of ~ 0.0019 USD to 1 CRC.

### Relevant error log or model output

```shell
Here are examples of the JSON for captures versus refunds (I removed all non-relevant fields, etc.):

capture

{
"charges": {
"data": [
{
"balance_transaction": {
"exchange_rate": 0.00184531
}
}
]
}
}
```

refund
```
{
"balance_transaction": {
"exchange_rate": 0.00187629,
"id": "txn_12345",
"object": "balance_transaction"
},
"charge": {
"balance_transaction": "txn_12345"
}
}
```
```

### Expected behavior

The expected behavior is to calculate the correct exchange rates, etc. for refund transactions.

### dbt Project configurations

```
name: chabi
version: 1.0.0
config-version: 2
profile: '{{ env_var(''DBT_PROFILE'', '''') }}'
model-paths:
- models
analysis-paths:
- analyses
test-paths:
- tests
seed-paths:
- seeds
macro-paths:
- macros
snapshot-paths:
- snapshots
target-path: target
clean-targets:
- target
- dbt_packages
models:
+persist_docs:
relation: true
columns: true
+sql_header: alter session set timezone = 'America/Los_Angeles'; alter session set week_start = 1; alter session set TIMESTAMP_TYPE_MAPPING = TIMESTAMP_LTZ;
ad_reporting:
+schema:
intermediate:
+schema: stg
klaviyo:
+schema:
intermediate:
+schema: stg
+materialized: table
klaviyo_source:
+schema: stg
shopify:
+schema:
shopify_source:
+schema: stg
shopify_holistic_reporting:
+schema:
intermediate:
+schema: stg
microsoft_ads:
+schema: stg
microsoft_ads_source:
+schema: stg
amazon_ads:
+schema: stg
amazon_ads_source:
+schema: stg
google_ads:
+schema: stg
google_ads_source:
+schema: stg
tiktok_ads:
+schema: stg
tiktok_ads_source:
+schema: stg
facebook_ads:
+schema: stg
facebook_ads_source:
+schema: stg
snapchat_ads:
+schema: stg
snapchat_ads_source:
+schema: stg
vars:
iana_timezone: America/Los_Angeles
snapchat_schema: SNAPCHAT_ADS
pinterest_schema:
amazon_ads_schema: AMAZON_ADS
google_ads_schema: GOOGLE_ADS
tiktok_ads_schema: TIKTOK_ADS
facebook_ads_schema: FACEBOOK_ADS
linkedin_ads_schema:
microsoft_ads_schema: BINGADS1
pinterest__using_keywords: false
ad_reporting__amazon_ads_enabled: true
ad_reporting__google_ads_enabled: true
ad_reporting__reddit_ads_enabled: false
ad_reporting__tiktok_ads_enabled: true
ad_reporting__twitter_ads_enabled: false
ad_reporting__facebook_ads_enabled: true
ad_reporting__linkedin_ads_enabled: false
ad_reporting__snapchat_ads_enabled: true
ad_reporting__microsoft_ads_enabled: true
ad_reporting__pinterest_ads_enabled: false
amazon_ads__portfolio_history_enabled: false
ad_reporting__apple_search_ads_enabled: false
klaviyo_schema: KLAVIYO
shopify_schema: SHOPIFY
```

### Package versions

```
packages:
- package: dbt-labs/codegen
version: 0.9.0
- package: dbt-labs/dbt_utils
version:
- '>=1.0.0'
- <2.0.0
- package: fivetran/ad_reporting
version: 1.5.0
- package: fivetran/klaviyo
version: 0.5.0
- package: fivetran/shopify
version: 0.8.1
- package: fivetran/shopify_holistic_reporting
version: 0.4.0
```

### What database are you using dbt with?

snowflake

### dbt Version

```
Core:
- installed: 1.5.0
- latest: 1.7.2 - Update available!

Your version of dbt-core is out of date!
You can find instructions for upgrading here:
https://docs.getdbt.com/docs/installation

Plugins:
- snowflake: 1.5.0 - Update available!

At least one plugin is out of date or incompatible with dbt-core.
You can find instructions for upgrading here:
https://docs.getdbt.com/docs/installation

```

### Additional Context

_No response_

### Are you willing to open a PR to help address this issue?

- [ ] Yes.
- [X] Yes, but I will need assistance and will schedule time during our [office hours](https://calendly.com/fivetran-solutions-team/fivetran-solutions-team-office-hours) for guidance
- [ ] No.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.