fivetran / fivetran/dbt_stripe

[Bug] Error in tmp models due to metadata type json

Open
#64 3 comments 0 reactions 0 assignees View on GitHub
type:wontfix
Dominant language
No language data
Stars
61
Forks
40
PR merge metrics
No merged PRs in 30d

Description

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Describe the issue

Tmp models returns an error because cannot cast metadata dimension which is passed as json type instead of string.
Metadata dimension used to be string but most recent Fivetran tables turned it into json.

### Relevant error log or model output

```shell
Database Error in model stg_stripe__invoice_line_item_tmp (models/tmp/stg_stripe__invoice_line_item_tmp.sql)
Invalid cast from JSON to STRING at [94:26]
compiled Code at target/run/stripe_source/models/tmp/stg_stripe__invoice_line_item_tmp.sql
```

### Expected behavior

The tmp models should cast metadata fields as string with `to_json()` function to convert the value to a JSON string before casting it to a string.

The package should generate the output tables without errors.

### dbt Project configurations

name: 'dbzaps'
version: '1.0.0'
config-version: 2

# This setting configures which "profile" dbt uses for this project.
profile: 'default'

# These configurations specify where dbt should look for different types of files.
# The `source-paths` config, for example, states that models in this project can be
# found in the "models/" directory. You probably won't need to change these!
model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

target-path: "target" # directory which will store compiled SQL files
clean-targets: # directories to be removed by `dbt clean`
- "target"
- "dbt_packages"

# Configuring models
# Full documentation: https://docs.getdbt.com/docs/configuring-models

models:
# Applies to all files under models/bilateralstimulation/
dbzaps:
bilateralstimulation:
staging:
materialized: view
intermediate:
materialized: ephemeral
marts:
+materialized: table

# Fivetran Stripe | Bilateral Stimulation
vars:
stripe_union_schemas: ['stripe_bilateralstimulation_de','stripe_bilateralstimulation_us','stripe_bilateralstimulation'] # use this if the data is in different schemas/datasets of the same database/project

### Package versions

packages:
- package: fivetran/stripe
version: 0.10.1

### What database are you using dbt with?

bigquery

### dbt Version

dbt Version: 1.4 (latest)

### Additional Context

Screenshot 2023-05-17 at 08 15 29

### Are you willing to open a PR to help address this issue?

- [X] Yes.
- [ ] Yes, but I will need assistance and will schedule time during our [office hours](https://calendly.com/fivetran-solutions-team/fivetran-solutions-team-office-hours) for guidance
- [ ] No.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the stg_stripe__invoice_line_item_tmp model under models/tmp and compare it with the compiled file at target/run/stripe_source/models/tmp/stg_stripe__invoice_line_item_tmp.sql. Reproduce the BigQuery error and inspect how metadata dimensions are handled across the tmp models. Done means the package generates the output tables without the JSON-to-STRING cast error.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, sql
Domain
data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.