Redshift historical retrieval query performance regression
Open
Community Contribution Needed
critical
kind/bug
priority/p1
- Dominant language
- Python
- Stars
- 7.3k
- Forks
- 1.4k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 15
Description
There seems to have been some regression in historical retrieval for Redshift that happened somewhere between 0.14.1 and 0.16.1. A quick summary of one particular user's issue:
- currently the point-in-time joins on big entity dataframes are slow in Redshift. The entity_df is generated right now by creating e.g. a `user_id` CROSS JOIN `item_id`
- The tables in question have usually have 2-4 columns per feature view, and models use between 10 - 30 feature views.
@woop had done some quick investigation and the main shift seemed to happen in https://github.com/feast-dev/feast/pull/1911 introduced by @MattDelac
Contributor guide
Assessment
This issue has not been assessed yet.