feldera / feldera/feldera

[dbsp] Micro/macro-step support for recursive circuits

Open
#4,110 0 comments 0 reactions 1 assignee View on GitHub

@ryzhyk is already working on this.

Since May 31, 2025.

DBSP core
Dominant language
Rust
Stars
2.1k
Forks
154
Avg merge
2d 19h
Merged PRs (30d)
133

Description

A recursive circuit operates as both an accumulator and a splitter. It accumulates changes produced by the outer circuit. Once the outer circuit's scheduler calls flush on the nested circuit, indicating that no more inputs will be received in the current macrostep, the recursive circuit starts iterating. Every iteration of the recursive circuit is also broken up into microsteps. The entire execution of the recursive circuit completes within one microstep of the outer circuit. The output is assembled into a spine (as it already is today), but instead of consolidating this spine into a single batch, we will output it chunk-by-chunk over multiple subsequent microsteps.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.