fivetran / fivetran/dbt_stripe
[Bug] Error in tmp models due to metadata type json
- Dominant language
- No language data
- Stars
- 61
- Forks
- 40
- PR merge metrics
- No merged PRs in 30d
Description
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Describe the issue
Tmp models returns an error because cannot cast metadata dimension which is passed as json type instead of string.
Metadata dimension used to be string but most recent Fivetran tables turned it into json.
### Relevant error log or model output
```shell
Database Error in model stg_stripe__invoice_line_item_tmp (models/tmp/stg_stripe__invoice_line_item_tmp.sql)
Invalid cast from JSON to STRING at [94:26]
compiled Code at target/run/stripe_source/models/tmp/stg_stripe__invoice_line_item_tmp.sql
```
### Expected behavior
The tmp models should cast metadata fields as string with `to_json()` function to convert the value to a JSON string before casting it to a string.
The package should generate the output tables without errors.
### dbt Project configurations
name: 'dbzaps'
version: '1.0.0'
config-version: 2
# This setting configures which "profile" dbt uses for this project.
profile: 'default'
# These configurations specify where dbt should look for different types of files.
# The `source-paths` config, for example, states that models in this project can be
# found in the "models/" directory. You probably won't need to change these!
model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]
target-path: "target" # directory which will store compiled SQL files
clean-targets: # directories to be removed by `dbt clean`
- "target"
- "dbt_packages"
# Configuring models
# Full documentation: https://docs.getdbt.com/docs/configuring-models
models:
# Applies to all files under models/bilateralstimulation/
dbzaps:
bilateralstimulation:
staging:
materialized: view
intermediate:
materialized: ephemeral
marts:
+materialized: table
# Fivetran Stripe | Bilateral Stimulation
vars:
stripe_union_schemas: ['stripe_bilateralstimulation_de','stripe_bilateralstimulation_us','stripe_bilateralstimulation'] # use this if the data is in different schemas/datasets of the same database/project
### Package versions
packages:
- package: fivetran/stripe
version: 0.10.1
### What database are you using dbt with?
bigquery
### dbt Version
dbt Version: 1.4 (latest)
### Additional Context
### Are you willing to open a PR to help address this issue?
- [X] Yes.
- [ ] Yes, but I will need assistance and will schedule time during our [office hours](https://calendly.com/fivetran-solutions-team/fivetran-solutions-team-office-hours) for guidance
- [ ] No.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the stg_stripe__invoice_line_item_tmp model under models/tmp and compare it with the compiled file at target/run/stripe_source/models/tmp/stg_stripe__invoice_line_item_tmp.sql. Reproduce the BigQuery error and inspect how metadata dimensions are handled across the tmp models. Done means the package generates the output tables without the JSON-to-STRING cast error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, sql
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100