fivetran / fivetran/dbt_fivetran_utils
Macros to add/remove
- Dominant language
- Shell
- Stars
- 33
- Forks
- 22
- PR merge metrics
- No merged PRs in 30d
Description
Starting to gather a housekeeping list of macros when we cut v0.5.0.
## Add
- Macros used in several packages
- Macros where we want to curtail divergence (i.e. packages have built their own local, near-duplicate versions instead of using the shared macro)
- Investigate if we can consolidate:
- `lookback`
- 8 packages (`dbt_mixpanel`, `dbt_amplitude`, `dbt_iterable`, `dbt_shopify`, `dbt_ga4_export`, `dbt_netsuite`, `dbt_jira`, `dbt_fivetran_log`) each carry an identical local `_lookback` macro. Confirmed byte-for-byte identical logic across all 8 (only differ by macro name and default `datepart`/`interval`/`safety_date` values).
- `extract_url_parameter`
- 10 packages (`dbt_facebook_ads`, `dbt_google_ads`, `dbt_linkedin`, `dbt_microsoft_ads`, `dbt_pinterest`, `dbt_reddit_ads`, `dbt_snapchat_ads`, `dbt_stackadapt`, `dbt_tiktok_ads`, `dbt_twitter`) have independently copy-pasted this macro under a package-prefixed name (e.g. `facebook_ads_extract_url_parameter`).
## Remove (move locally to packages)
- Macros used in <2 or fewer packages (count tbd) unless expected to increase.
- Macros where the `dbt_utils`/`dbt` equivalent is sufficient (mostly the cross-database compatibility macros below — `dbt-core` now ships its own cross-db macros, e.g. `dbt.try_cast`, which most packages call directly instead of ours).
### Used in 0 packages (Consider removal)
| Macro | Notes |
| -- | -- |
| add_dbt_source_relation | |
| ceiling | |
| collect_freshness | |
| drop_schemas_automation | |
| dummy_coalesce_value | |
| extract_url_parameter | see "Add" above - packages reimplemented locally instead of adopting; candidate to fix adoption rather than remove |
| first_value | cross-db compat - `dbt`/`dbt_utils` equivalent likely sufficient |
| json_extract | |
| seed_data_helper | obsolete CI/integration-test tooling only |
| snowflake_seed_data | obsolete CI/integration-test tooling only |
| wrap_in_quotes | |
### Used in exactly 1 package (Move locally to package)
| Macro | Package(s) |
| -- | -- |
| array_agg | dbt_hubspot |
| calculated_fields | dbt_hubspot |
| empty_variable_warning | dbt_zendesk |
| fivetran_date_spine | dbt_fivetran_log |
| remove_prefix_from_columns | dbt_hubspot |
| timestamp_add | dbt_zendesk |
| timestamp_diff | dbt_qualtrics |
| union_relations | dbt_marketo |
### Used in exactly 2 packages (Consider moving locally to package)
| Macro | Packages |
| -- | -- |
| enabled_vars_one_true | dbt_quickbooks, dbt_sage_intacct |
| try_cast | dbt_klaviyo, dbt_shopify_holistic_reporting |
### Reference - 3–5 packages, but probably worth keeping in fivetran_utils)
| Macro | Packages |
| -- | -- |
| max_bool | dbt_qualtrics, dbt_shopify, dbt_zendesk |
| percentile | dbt_intercom, dbt_jira, dbt_qualtrics |
| pivot_json_extract | dbt_amplitude, dbt_mixpanel, dbt_stripe |
| json_parse | dbt_fivetran_log, dbt_klaviyo, dbt_shopify, dbt_youtube_analytics, dbt_zendesk |
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by auditing the listed macros and their package usage, beginning with the duplicated lookback and extract_url_parameter implementations across the named packages. Review the zero-, one-, and two-package tables alongside the reference list, then document which macros should be shared, removed, or moved locally. Done means the v0.5.0 housekeeping list has confirmed decisions and no unresolved usage counts.
Written by the indexing model from the issue text.
Assessment
- Domain
- data-engineering
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100