fivetran / fivetran/dbt_salesforce

[Bug] column_list_to_dict macro will override the column's metadata if you pass through a column that is already selected from the source.

Open
#72 5 comments 0 reactions 1 assignee Claimed by @fivetran-catfritz View on GitHub
type:wontfix
Dominant language
No language data
Stars
53
Forks
38
PR merge metrics
No merged PRs in 30d

Description

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Describe the issue

For example, if we configure pass-through column email with a different alias
```yml
salesforce__lead_pass_through_columns: # pass columns from lead source
- name: "email"
alias: "internal_email"
renamed_column_name: "Email"
transform_sql: "case when email like '%domain%' then true else false end"
```

[This line](https://github.com/fivetran/dbt_salesforce_source/blob/48d32b46c5c8aa2598ceee1d621d15cbf7e4cbc3/models/salesforce/stg_salesforce__lead.sql#L5) call `column_list_to_dict` macro, that will override the existing email's metadata, therefore, cause `renamed_column_name` field empty, result in invalid SQL syntax (rendered by `salesforce_source.coalesce_rename` macro):
```sql
with fields as (
select ...
),
final as (
select
...
coalesce(cast( as TEXT),
cast(email as TEXT))
as internal_email,
from fields
)
select *
from final
where not coalesce(is_deleted, false)
);
```

### Relevant error log or model output

```shell
syntax error at or near "as" in context "(cast( as", at line 793, column 20
compiled code at target/run/salesforce_source/models/salesforce/stg_salesforce__lead.sql
```

### Expected behavior

Should not override metadata of existing column,

expected:
```sql
coalesce(cast(Email as TEXT),
cast(email as TEXT))
as email,
```

### Possible solution

Add the field `{is_rename: True}` for pass-through columns in `get_*_columns.sql` or within the `fivetran_utils.add_pass_through_columns` macro so that the macro `column_list_to_dict` does not override them. Keen to hear your suggestions.

### dbt Project configurations

```yml
salesforce__lead_pass_through_columns: # pass columns from lead source
- name: "email"
alias: "internal_email"
renamed_column_name: "Email"
transform_sql: "case when email like '%domain%' then true else false end"
```

### Package versions

">=1.1.0", "<1.2.0"

### What database are you using dbt with?

redshift

### How are you running this dbt package?

dbt Core™

### dbt Version

1.8

### Additional Context

_No response_

### Are you willing to open a PR to help address this issue?

- [ ] Yes.
- [X] Yes, but I will need assistance.
- [ ] No.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.