feast-dev / feast-dev/feast

Redshift historical retrieval query performance regression

Open
#2,222 6 comments 0 reactions 0 assignees View on GitHub
Community Contribution Needed critical kind/bug priority/p1
Dominant language
Python
Stars
7.3k
Forks
1.4k
Avg merge
1d 21h
Merged PRs (30d)
15

Description

There seems to have been some regression in historical retrieval for Redshift that happened somewhere between 0.14.1 and 0.16.1. A quick summary of one particular user's issue:

- currently the point-in-time joins on big entity dataframes are slow in Redshift. The entity_df is generated right now by creating e.g. a `user_id` CROSS JOIN `item_id`
- The tables in question have usually have 2-4 columns per feature view, and models use between 10 - 30 feature views.

@woop had done some quick investigation and the main shift seemed to happen in https://github.com/feast-dev/feast/pull/1911 introduced by @MattDelac

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.