fivetran / fivetran/dbt_zuora

Not all models include the enabled config

Open
#32 4 comments 0 reactions 0 assignees View on GitHub
type:enhancement
Dominant language
No language data
Stars
3
Forks
1
PR merge metrics
No merged PRs in 30d

Description

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Describe the issue

Currently, the package expects certain sources to be available. Some models, but not all, include the enabled config, which makes it possible to skip the model if the source is not available. For example, `stg_zuora__credit_balance_adjustment` includes the following: `{{ config(enabled=var('zuora__using_credit_balance_adjustment', true)) }}` (code [here](https://github.com/fivetran/dbt_zuora_source/blob/main/models/stg_zuora__credit_balance_adjustment.sql#L1)). Of the 19 models, only 4 can be disabled:
- stg_zuora__credit_balance_adjustment
- stg_zuora__refund
- stg_zuora__refund_invoice_payment
- stg_zuora__taxation_item

At Pleo, we are not ingesting all the sources that are expected by the package, so even when leveraging the enabled config in the models where it's available, we are still left with models raising errors.

With the changes in [this PR](https://github.com/fivetran/dbt_zuora_source/pull/12), it is possible to disable any model, not just the subset listed above. Having the ability to disable models for which the sources are not ingested via Fivetran is a great feature, and it opens up use of this package even when only a few Fivetran sources are available.

### Relevant error log or model output

_No response_

### Expected behavior

I would expect to be able to leverage the package for as many models as we currently ingest via Fivetran. IOW, I would expect to be able to configure each model to be enabled.

Before making the changes, we had four models that raised errors because the sources are not available. I ran `dbt build --select +zuora_source` and the following shows the error:

![Screenshot 2024-02-26 at 16 08 21](https://github.com/fivetran/dbt_zuora/assets/117174676/2688b06b-3ed3-462f-9599-80cc9cf8cdb5)

After making the changes and disabling the models for sources we do not ingest, I ran `dbt build --select +zuora_source` again; no errors were raised.

![Screenshot 2024-02-27 at 16 06 12](https://github.com/fivetran/dbt_zuora/assets/117174676/38be3b10-c8cb-439f-8d3d-b5c897a69be5)

![Screenshot 2024-02-27 at 16 04 27](https://github.com/fivetran/dbt_zuora/assets/117174676/dbf9b428-b59b-4482-836d-2bce5d262fcf)

I then removed one of the new variables to ensure that it still correctly raised an error; it did:

![Screenshot 2024-02-27 at 16 08 12](https://github.com/fivetran/dbt_zuora/assets/117174676/e702d54d-e46f-4447-9b5e-5cc775810721)
Note: `zuora__using_contact: false` is commented out.

![Screenshot 2024-02-27 at 16 09 10](https://github.com/fivetran/dbt_zuora/assets/117174676/9a5eb5b3-844a-430a-9266-8cd97427b27b)

### dbt Project configurations

```
zuora__using_credit_balance_adjustment: false
zuora__using_refund: false
zuora__using_refund_invoice_payment: false
zuora__using_taxation_item: false
```

### Package versions

```
- package: fivetran/zuora_source
version: [">=0.2.0", "<0.3.0"]
```

### What database are you using dbt with?

bigquery

### dbt Version

![Screenshot 2024-02-28 at 15 59 01](https://github.com/fivetran/dbt_zuora/assets/117174676/77b76707-8a28-4571-ad0b-c316e267966b)

### Additional Context

I made and tested the changes in [this PR](https://github.com/fivetran/dbt_zuora_source/pull/12).

### Are you willing to open a PR to help address this issue?

- [X] Yes.
- [ ] Yes, but I will need assistance and will schedule time during our [office hours](https://calendly.com/fivetran-solutions-team/fivetran-solutions-team-office-hours) for guidance
- [ ] No.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.