feast-dev / feast-dev/feast

Support for s3_url_style='path' Configuration in DuckDB Offline Store

Open
#5,597 1 comment 0 reactions 0 assignees View on GitHub
kind/feature
Dominant language
Python
Stars
7.3k
Forks
1.4k
Avg merge
1d 21h
Merged PRs (30d)
15

Description

**Is your feature request related to a problem? Please describe.**
When using DuckDB as an offline store with S3-compatible storage, authentication can fail if the S3 URL style is not set correctly. DuckDB requires the configuration `s3_url_style='path'` for certain S3 providers, but this setting is not currently supported via environment variables: [https://duckdb.org/docs/stable/core_extensions/httpfs/s3api_legacy_authentication.html#configuration](url)

**Describe the solution you'd like**
Enable Feast to pass the `s3_url_style='path'` configuration to DuckDB during initialization. This would allow users to work with S3-compatible storage providers that require path-style URLs.

**Describe alternatives you've considered**
A temporary workaround involves manually adding the following code line:
`self.con.sql("SET s3_url_style='path';")`
at line 415 in the file:
`AppData\Local\Programs\Python\Python312\Lib\site-packages\ibis\backends\duckdb\__init__.py`
However, this is not a sustainable or clean solution, especially for production environments.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.