feast-dev / feast-dev/feast

dbt integration: Validate entity column data type is appropriate

Open
#5,876 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.3k
Forks
1.4k
Avg merge
1d 21h
Merged PRs (30d)
15

Description

## Context
PR #5827 added dbt integration that creates Entity objects from dbt model columns.

## Problem
No validation that the entity column has an appropriate data type for use as an entity key. Entity keys should typically be:
- STRING / VARCHAR
- INT / INT64 / BIGINT
- UUID (if supported)

But the code would accept any column type including:
- FLOAT / DOUBLE (non-deterministic for joins)
- BYTES (not suitable for entity keys)
- TIMESTAMP (rarely appropriate)

## Current Behavior
```python
# In dbt_import.py:191-197
if entity_column not in column_names:
click.echo(warning)
continue
# No type checking!
```

## Proposed Solution
Add validation and warning:

```python
entity_col = next((c for c in model.columns if c.name == entity_column), None)
if entity_col:
normalized_type = entity_col.data_type.upper()
valid_entity_types = ['STRING', 'TEXT', 'VARCHAR', 'INT', 'INT32', 'INT64', 'INTEGER', 'BIGINT', 'UUID']

if not any(t in normalized_type for t in valid_entity_types):
click.echo(
f"{Fore.YELLOW}Warning: Entity column '{entity_column}' has type "
f"'{entity_col.data_type}' which may not be suitable for entity keys."
f" Recommended types: STRING, INT64{Style.RESET_ALL}"
)
```

## Edge Cases to Handle
- FLOAT columns (should warn strongly)
- ARRAY columns (invalid for entities)
- Complex/nested types (invalid)

## Related
- PR #5827

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.