fivetran / fivetran/dbt_salesforce_formula_utils

[Feature] Add option to remove hard-coded where clause from sfdc_formula_view

Open
#104 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

type:wontfix
Dominant language
Shell
Stars
18
Forks
21
Avg merge
15d 20h
Merged PRs (30d)
2

Description

Is there an existing feature request for this?
  • I have searched the existing issues
Describe the Feature

Currently, fivetran is populating a where clause in fivetran_formula_model.model which is hard-coding the removal of deleted records.
WHERE NOT mt."_FIVETRAN_DELETED"

Our company needs to remove this criteria from all of our Salesforce models so we ingest deleted items into our DBT snapshots. I propose adding that functionality directly to the sfdc_formula_view macro:

{% if remove_where_clause %}
    {{ table_results[0]|replace("WHERE NOT mt.\"_FIVETRAN_DELETED\"", "") }}
{% else %}
    {{ table_results[0] }}
{% endif %}
Describe alternatives you've considered

There's a few ways to solve this problem including building a view on top of the existing fivetran_formula_model table that makes this replacement. It's easy enough to update the schema name in the salesforce source yml (which is what we've gone with for the time being). However, I feel like this is functionality that is useful for not just our team and the solution that we've implemented is a bit clunky.

Are you interested in contributing this feature?
  • Yes.
  • Yes, but I will need assistance and will schedule time during your office hours for guidance.
  • No.
Anything else?

I have this built out in a branch already and it functions as I would expect it in testing:
https://github.com/paperlinguist/dbt_salesforce_formula_utils/blob/paperlinguist-add-where-clause-removal/macros/sfdc_formula_view.sql

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with macros/sfdc_formula_view.sql and compare it with the contributor's paperlinguist-add-where-clause-removal branch. Verify the option's intended behavior in dbt: the hard-coded _FIVETRAN_DELETED filter is omitted when enabled and remains unchanged by default; confirm both paths using the package's existing testing workflow.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
database
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.