fivetran / fivetran/dbt_salesforce
[Bug] column_list_to_dict macro will override the column's metadata if you pass through a column that is already selected from the source.
- Dominant language
- No language data
- Stars
- 53
- Forks
- 38
- PR merge metrics
- No merged PRs in 30d
Description
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Describe the issue
For example, if we configure pass-through column email with a different alias
```yml
salesforce__lead_pass_through_columns: # pass columns from lead source
- name: "email"
alias: "internal_email"
renamed_column_name: "Email"
transform_sql: "case when email like '%domain%' then true else false end"
```
[This line](https://github.com/fivetran/dbt_salesforce_source/blob/48d32b46c5c8aa2598ceee1d621d15cbf7e4cbc3/models/salesforce/stg_salesforce__lead.sql#L5) call `column_list_to_dict` macro, that will override the existing email's metadata, therefore, cause `renamed_column_name` field empty, result in invalid SQL syntax (rendered by `salesforce_source.coalesce_rename` macro):
```sql
with fields as (
select ...
),
final as (
select
...
coalesce(cast( as TEXT),
cast(email as TEXT))
as internal_email,
from fields
)
select *
from final
where not coalesce(is_deleted, false)
);
```
### Relevant error log or model output
```shell
syntax error at or near "as" in context "(cast( as", at line 793, column 20
compiled code at target/run/salesforce_source/models/salesforce/stg_salesforce__lead.sql
```
### Expected behavior
Should not override metadata of existing column,
expected:
```sql
coalesce(cast(Email as TEXT),
cast(email as TEXT))
as email,
```
### Possible solution
Add the field `{is_rename: True}` for pass-through columns in `get_*_columns.sql` or within the `fivetran_utils.add_pass_through_columns` macro so that the macro `column_list_to_dict` does not override them. Keen to hear your suggestions.
### dbt Project configurations
```yml
salesforce__lead_pass_through_columns: # pass columns from lead source
- name: "email"
alias: "internal_email"
renamed_column_name: "Email"
transform_sql: "case when email like '%domain%' then true else false end"
```
### Package versions
">=1.1.0", "<1.2.0"
### What database are you using dbt with?
redshift
### How are you running this dbt package?
dbt Core™
### dbt Version
1.8
### Additional Context
_No response_
### Are you willing to open a PR to help address this issue?
- [ ] Yes.
- [X] Yes, but I will need assistance.
- [ ] No.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.