fivetran / fivetran/dbt_stripe
[FEATURE] metadata columns does not appear summary tables
- Dominant language
- No language data
- Stars
- 61
- Forks
- 40
- PR merge metrics
- No merged PRs in 30d
Description
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Describe the issue
I want to add some columns from the metadata column into some of the summary tables:
stripe__balance_transactions
stripe__invoice_details
stripe__invoice_line_item_details
stripe__subscription_details
stripe__customer_overview
With the exception of `stripe__customer_overview` adding the following vars in `dbt_project.yml` did not help:
```
stripe__charge_metadata: ['col1', 'col2', 'col3', 'col4']
stripe__invoice_metadata: ['col1', 'col2', 'col3', 'col4']
stripe__subscription_metadata: ['col1', 'col2', 'col3', 'col4']
stripe__customer_metadata: ['col1']
```
For now, I did the same as [here](https://github.com/fivetran/dbt_stripe/issues/43) , adding manually
```
{% if var('stripe__charge_metadata',[]) %}
{% for metadata in var('stripe__charge_metadata') %}
,charge.{{ metadata }} as charge_{{ metadata }}
{% endfor %}
{% endif %}
{% if var('stripe__customer_metadata',[]) %}
{% for metadata in var('stripe__customer_metadata') %}
,customer.{{ metadata }} as customer_{{ metadata }}
{% endfor %}
{% endif %}
{% if var('stripe__charge_metadata',[]) %}
{% for metadata in var('stripe__charge_metadata') %}
,refund_charge.{{ metadata }} as refund_charge_{{ metadata }}
{% endfor %}
{% endif %}
```
To files like `stripe__balance_transactions.sql`
### Relevant error log or model output
```shell
No error was reported as dbt run and test work.
```
### Expected behavior
Columns `col1, col2, col3, col4` appear in:
stripe__balance_transactions
stripe__invoice_details
stripe__invoice_line_item_details
stripe__subscription_details
Column `col1` appears in (it is):
stripe__customer_overview
### dbt Project configurations
```
# Name your project! Project names should contain only lowercase characters
# and underscores. A good package name should reflect your organization's
# name or the intended use of these models
name: 'fivetran_dbt'
version: '1.0.0'
config-version: 2
# This setting configures which "profile" dbt uses for this project.
profile: 'fivetran_dbt'
# These configurations specify where dbt should look for different types of files.
# The `model-paths` config, for example, states that models in this project can be
# found in the "models/" directory. You probably won't need to change these!
model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]
target-path: "target" # directory which will store compiled SQL files
clean-targets: # directories to be removed by `dbt clean`
- "target"
- "dbt_packages"
# Configuring models
# Full documentation: https://docs.getdbt.com/docs/configuring-models
# In this example config, we tell dbt to build all models in the example/ directory
# as tables. These settings can be overridden in the individual model files
# using the `{{ config(...) }}` macro.
models:
fivetran_dbt:
stripe:
stripe_source:
vars:
using_schedules: False
stripe_source:
stripe_database: dkall
stripe_schema: stripe
stripe_database: dkall
stripe_schema: stripe
stripe__subscription_history: True
stripe_source:
stripe__using_invoice_line_sub_filter: false
stripe__charge_metadata: ['col1', 'col2', 'col3', 'col4']
stripe__invoice_metadata: ['col1', 'col2', 'col3', 'col4']
stripe__subscription_metadata: ['col1', 'col2', 'col3', 'col4']
stripe__customer_metadata: ['col1']
```
### Package versions
```
packages:
- package: fivetran/stripe
version: [">=0.10.0", "<0.11.0"]
```
### What database are you using dbt with?
redshift
### dbt Version
```
Core:
- installed: 1.5.1
- latest: 1.5.1 - Up to date!
Plugins:
- postgres: 1.5.1 - Up to date!
- redshift: 1.5.4 - Up to date!
```
### Additional Context
_No response_
### Are you willing to open a PR to help address this issue?
- [ ] Yes.
- [ ] Yes, but I will need assistance and will schedule time during our [office hours](https://calendly.com/fivetran-solutions-team/fivetran-solutions-team-office-hours) for guidance
- [X] No.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with stripe__balance_transactions.sql and trace the models producing the other listed summary tables. Compare their metadata handling with the approach referenced in dbt_stripe issue 43, then verify the configured metadata columns appear in the expected tables when dbt runs and tests successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sql
- Domain
- data, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100