fivetran / fivetran/dbt_stripe

[FEATURE] metadata columns does not appear summary tables

Open
#65 5 comments 0 reactions 0 assignees View on GitHub
type:enhancement
Dominant language
No language data
Stars
61
Forks
40
PR merge metrics
No merged PRs in 30d

Description

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Describe the issue

I want to add some columns from the metadata column into some of the summary tables:

stripe__balance_transactions
stripe__invoice_details
stripe__invoice_line_item_details
stripe__subscription_details
stripe__customer_overview

With the exception of `stripe__customer_overview` adding the following vars in `dbt_project.yml` did not help:

```
stripe__charge_metadata: ['col1', 'col2', 'col3', 'col4']
stripe__invoice_metadata: ['col1', 'col2', 'col3', 'col4']
stripe__subscription_metadata: ['col1', 'col2', 'col3', 'col4']
stripe__customer_metadata: ['col1']
```

For now, I did the same as [here](https://github.com/fivetran/dbt_stripe/issues/43) , adding manually

```
{% if var('stripe__charge_metadata',[]) %}
{% for metadata in var('stripe__charge_metadata') %}
,charge.{{ metadata }} as charge_{{ metadata }}
{% endfor %}
{% endif %}

{% if var('stripe__customer_metadata',[]) %}
{% for metadata in var('stripe__customer_metadata') %}
,customer.{{ metadata }} as customer_{{ metadata }}
{% endfor %}
{% endif %}

{% if var('stripe__charge_metadata',[]) %}
{% for metadata in var('stripe__charge_metadata') %}
,refund_charge.{{ metadata }} as refund_charge_{{ metadata }}
{% endfor %}
{% endif %}
```

To files like `stripe__balance_transactions.sql`

### Relevant error log or model output

```shell
No error was reported as dbt run and test work.
```

### Expected behavior

Columns `col1, col2, col3, col4` appear in:

stripe__balance_transactions
stripe__invoice_details
stripe__invoice_line_item_details
stripe__subscription_details

Column `col1` appears in (it is):
stripe__customer_overview

### dbt Project configurations

```
# Name your project! Project names should contain only lowercase characters
# and underscores. A good package name should reflect your organization's
# name or the intended use of these models
name: 'fivetran_dbt'
version: '1.0.0'
config-version: 2

# This setting configures which "profile" dbt uses for this project.
profile: 'fivetran_dbt'

# These configurations specify where dbt should look for different types of files.
# The `model-paths` config, for example, states that models in this project can be
# found in the "models/" directory. You probably won't need to change these!
model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

target-path: "target" # directory which will store compiled SQL files
clean-targets: # directories to be removed by `dbt clean`
- "target"
- "dbt_packages"

# Configuring models
# Full documentation: https://docs.getdbt.com/docs/configuring-models

# In this example config, we tell dbt to build all models in the example/ directory
# as tables. These settings can be overridden in the individual model files
# using the `{{ config(...) }}` macro.
models:
fivetran_dbt:
stripe:
stripe_source:

vars:
using_schedules: False
stripe_source:
stripe_database: dkall
stripe_schema: stripe
stripe_database: dkall
stripe_schema: stripe
stripe__subscription_history: True
stripe_source:
stripe__using_invoice_line_sub_filter: false
stripe__charge_metadata: ['col1', 'col2', 'col3', 'col4']
stripe__invoice_metadata: ['col1', 'col2', 'col3', 'col4']
stripe__subscription_metadata: ['col1', 'col2', 'col3', 'col4']
stripe__customer_metadata: ['col1']
```

### Package versions

```
packages:
- package: fivetran/stripe
version: [">=0.10.0", "<0.11.0"]
```

### What database are you using dbt with?

redshift

### dbt Version

```
Core:
- installed: 1.5.1
- latest: 1.5.1 - Up to date!

Plugins:
- postgres: 1.5.1 - Up to date!
- redshift: 1.5.4 - Up to date!

```

### Additional Context

_No response_

### Are you willing to open a PR to help address this issue?

- [ ] Yes.
- [ ] Yes, but I will need assistance and will schedule time during our [office hours](https://calendly.com/fivetran-solutions-team/fivetran-solutions-team-office-hours) for guidance
- [X] No.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with stripe__balance_transactions.sql and trace the models producing the other listed summary tables. Compare their metadata handling with the approach referenced in dbt_stripe issue 43, then verify the configured metadata columns appear in the expected tables when dbt runs and tests successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
data, databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.