feast-dev / feast-dev/feast

Add DynamoDB Accelerator (DAX) support for sub-millisecond reads

Open
#6,059 1 comment 0 reactions 0 assignees View on GitHub
kind/feature
Dominant language
Python
Stars
7.3k
Forks
1.4k
Avg merge
3d 16h
Merged PRs (30d)
15

Description

**Is your feature request related to a problem? Please describe.**

DynamoDB online store latency is fundamentally limited by network round-trip time to the DynamoDB service (~50-80ms per request). Even with all code optimizations applied, achieving <60ms p99 latency for production workloads (50 entities × 200 features) is not possible with direct DynamoDB access.
Benchmarks results here shows: https://github.com/abhijeet-dhumal/featurestore-benchmarks/tree/perf-with-6optimisations/fs-online-benchmark/locust-bechmarking/feast-benchmarking
- Current latency: 169ms p99 at 50 entities × 200 features
- Target SLA: 60ms p99
- Gap: 109ms that cannot be closed with code optimizations alone

This is frustrating for users who need low-latency feature serving for real-time ML inference but want to use DynamoDB for its scalability and managed infrastructure.

**Describe the solution you'd like**
Add native support for DynamoDB Accelerator (DAX) - AWS's in-memory caching layer for DynamoDB. This would allow users to configure a DAX endpoint for sub-millisecond cached reads:
```
online_store:
type: dynamodb
region: us-east-1
use_dax: true
endpoint_url: "daxs://my-cluster.abc123.dax-clusters.us-east-1.amazonaws.com:8111"
```
When enabled, Feast would use amazon-dax-client instead of boto3 for read operations, providing:

- <1ms latency for cached reads (vs 50-80ms for DynamoDB)
- 10x+ throughput improvement
- Transparent caching with no application changes

**Describe alternatives you've considered**

1. VPC Endpoints: Reduces latency by ~10-20ms but still limited by DynamoDB service latency.
2. Application-level caching (Redis sidecar): Adds operational complexity, requires cache invalidation logic, and duplicates the online store concept.
3. Switch to Redis online store: Achieves lower latency (~102ms at same scale) but loses DynamoDB's serverless scaling, managed infrastructure, and AWS integration benefits.
4. Reduce batch size: Requesting fewer entities per call meets latency benchmark but increases total request count and doesn't solve the underlying latency issue.
5. ElastiCache for DynamoDB: Not a direct integration; requires custom implementation.

None of these alternatives provide the combination of low latency + DynamoDB's managed infrastructure that DAX offers natively.

| Configuration | P99 Latency (50 entities × 200 features) |
|---------------|------------------------------------------|
| DynamoDB (current Feast master branch) | 169ms |
| DynamoDB + VPC Endpoint | ~150ms *(expected)* |
| DynamoDB + DAX | ~40ms *(expected)* |

#### Implementation scope:

- Add use_dax: bool config option
- Conditional client initialization using amazon-dax-client
- Optional dependency in pyproject.toml
- No changes to read/write API signatures

#### DAX characteristics:

- Fully API-compatible with DynamoDB for reads
- Write-through cache (writes go to both DAX and DynamoDB)
- Cost: ~$0.25/hr for dax.r5.large node
- Runs inside VPC (same network as application pods)

References:
- [AWS DAX Documentation](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DAX.html)
- [amazon-dax-client on PyPI](https://pypi.org/project/amazon-dax-client/)
- Benchmark [results](https://github.com/abhijeet-dhumal/featurestore-benchmarks/tree/perf-with-6optimisations/fs-online-benchmark/locust-bechmarking/feast-benchmarking) from Feast online store performance testing

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.