fivetran / fivetran/dbt_hubspot

[Feature] Support env_var usage to declare hubspot_database

Open
#178 3 comments 0 reactions 0 assignees View on GitHub
status:scoping type:enhancement
Dominant language
No language data
Stars
46
Forks
47
PR merge metrics
No merged PRs in 30d

Description

### Is there an existing feature request for this?

- [X] I have searched the existing issues

### Describe the Feature

Currently, the source hubspot database can be specified for the dbt hubspot packages as follows:

```
vars:
hubspot_database: your_destination_name
hubspot_schema: your_schema_name
```

We have a non-production version of hubspot and a production version of hubspot which are both synced by Fivetran into separate databases.

Currently [Jinja is not supported within the dbt vars config](https://docs.getdbt.com/docs/build/project-variables#defining-variables-in-dbt_projectyml).

As a result, dynamically selecting the correct hubspot database based on environment is not easily achievable. Users would have to override dbt variables with environment variables through the CLI.

**Feature**: Support env_var usage to declare hubspot_database
**Scenario**: dbt users have nonprod and production instances of hubspot
- **Given** A nonprod source and a production source exists for hubspot
- And they are in separate databases
- And I have not declared a static hubspot_database dbt var
- **When** I run a dbt project
- **Then** I should select the correct database depending on my environment

### How would you implement this feature?

In src_hubspot.yml [here](https://github.com/fivetran/dbt_hubspot_source/blob/5a4301c4a353f466b485aafd4c0de526ad495ed3/models/src_hubspot.yml#L6C5-L6C104)

Replacing

```
sources:
- name: hubspot
schema: "{{ var('hubspot_schema', 'hubspot') }}"
database: "{% if target.type != 'spark'%}{{ var('hubspot_database', target.database) }}{% endif %}"
```

with something that makes use of env_var. Perhaps:

```
sources:
- name: hubspot
schema: "{{ var('hubspot_schema', 'hubspot') }}"
database: "{% if target.type != 'spark'%}{{ var('hubspot_database', env_var('HUBSPOT_DATABASE', target.database)) }}{% endif %}"
```

### Describe alternatives you've considered

Alternative solutions include

- Parsing environment variables and overriding dbt variables through CLI
- Wrapping dbt variables and replacing with environment variables through scripts
- Creating separate dbt projects for each hubspot instance to be able to dynamically select sources in downstream projects

### Are you interested in contributing this feature?

- [ ] Yes.
- [X] Yes, but I will need assistance.
- [ ] No.

### Anything else?

_No response_

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in models/src_hubspot.yml at the HubSpot source declaration and review how dbt resolves env_var alongside the existing var fallback. Confirm the source database can be selected for non-production and production environments, while preserving static hubspot_database behavior and the Spark condition.

Written by the indexing model from the issue text.

Assessment

Domain
data-engineering, databases
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.