feast-dev / feast-dev/feast

dbt integration: Add support for data source connection configuration

Open
#5,875 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.3k
Forks
1.4k
Avg merge
1d 21h
Merged PRs (30d)
15

Description

## Context
PR #5827 added dbt integration with basic data source support.

## Problem
The generated data sources use default/inferred configurations but don't allow specifying:
- **BigQuery**: GCP project, dataset overrides
- **Snowflake**: account, warehouse, role, connection parameters
- **File**: base path, file format options

This means generated sources may not connect to the actual data.

## Current Limitation
```python
# BigQuery example - no way to override project
return BigQuerySource(
name=f"{model.name}_source",
table=model.full_table_name, # Uses dbt's database.schema.table
# Missing: project override, dataset override
)
```

## Proposed Enhancement

Add CLI options for data source configuration:

```bash
# BigQuery
feast dbt import -m manifest.json -e driver_id \
--data-source-type bigquery \
--bigquery-project my-gcp-project

# Snowflake
feast dbt import -m manifest.json -e driver_id \
--data-source-type snowflake \
--snowflake-account my-account \
--snowflake-warehouse my-warehouse \
--snowflake-role my-role

# File
feast dbt import -m manifest.json -e driver_id \
--data-source-type file \
--file-format parquet \
--file-base-path /data/warehouse
```

Or support reading from feature_store.yaml:
```yaml
offline_store:
type: bigquery
project: my-gcp-project

dbt:
inherit_offline_store_config: true
```

## Related
- PR #5827

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.