fivetran / fivetran/dbt_salesforce_formula_utils
[Feature] Add option to remove hard-coded where clause from sfdc_formula_view
Nobody has claimed this yet.
- Dominant language
- Shell
- Stars
- 18
- Forks
- 21
- Avg merge
- 15d 20h
- Merged PRs (30d)
- 2
Description
Is there an existing feature request for this?
- I have searched the existing issues
Describe the Feature
Currently, fivetran is populating a where clause in fivetran_formula_model.model which is hard-coding the removal of deleted records.
WHERE NOT mt."_FIVETRAN_DELETED"
Our company needs to remove this criteria from all of our Salesforce models so we ingest deleted items into our DBT snapshots. I propose adding that functionality directly to the sfdc_formula_view macro:
{% if remove_where_clause %}
{{ table_results[0]|replace("WHERE NOT mt.\"_FIVETRAN_DELETED\"", "") }}
{% else %}
{{ table_results[0] }}
{% endif %}
Describe alternatives you've considered
There's a few ways to solve this problem including building a view on top of the existing fivetran_formula_model table that makes this replacement. It's easy enough to update the schema name in the salesforce source yml (which is what we've gone with for the time being). However, I feel like this is functionality that is useful for not just our team and the solution that we've implemented is a bit clunky.
Are you interested in contributing this feature?
- Yes.
- Yes, but I will need assistance and will schedule time during your office hours for guidance.
- No.
Anything else?
I have this built out in a branch already and it functions as I would expect it in testing:
https://github.com/paperlinguist/dbt_salesforce_formula_utils/blob/paperlinguist-add-where-clause-removal/macros/sfdc_formula_view.sql
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with macros/sfdc_formula_view.sql and compare it with the contributor's paperlinguist-add-where-clause-removal branch. Verify the option's intended behavior in dbt: the hard-coded _FIVETRAN_DELETED filter is omitted when enabled and remains unchanged by default; confirm both paths using the package's existing testing workflow.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sql
- Domain
- database
- Issue type
- Feature
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100