apple / apple/foundationdb

Implement Changestreams

Open
#2,076 5 comments 5 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

In examining some potential layers for FDB, there's a number of APIs that offer something like "asychronously receive all updates for key K or key range K_begin to K_end in the order that they happened, and receive a notification if the connection breaks". There isn't currently a good way to implement this on FDB, as watches don't provide updates in order, they only provide that a watch happened.

Hallway discussions on this topic have resulted in that creating a stateless process that consumes the mutation stream from TLogs, and brokers out the relevant mutations to subscribing clients, should fulfill the requirements in a way that scales and be somewhat reasonable to implement.

Contributor guide

Open the contributing guide

Research direction

Start by examining the TLog mutation stream and the existing watch behavior described in the issue, then map the proposed stateless broker to the relevant FoundationDB subsystems. Done means subscribers can receive ordered updates for a key or key range and are notified when the connection breaks, with the design able to scale as discussed.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.