daphne-project / daphne-project/daphne
Calculate multiple results at once in DaphneLib
- Dominant language
- C++
- Stars
- 81
- Forks
- 83
- PR merge metrics
- No merged PRs in 30d
Description
DaphneLib is lazily evaluated, i.e., matrix/frame operations and control flow are not immediately executed, but build up a DAG of operations internally. This DAG is translated to a DaphneDSL script, which then goes through DAPHNE's entire parser/compiler/runtime stack, including all optimizations. To trigger the execution, one must say `compute()` on a DAG node in DaphneLib/Python.
The problems is that this always calculates just one result (can be a scalar/matrix/frame) including all operations that lead to this result, but:
1. Additional operations without results (e.g., `print()`, `write()`, `registerView()`, ... from DaphneDSL) are not included into the generated DaphneDSL script, since they are not on a path from the result to compute to the source nodes of the DAG. One particular use case of this would be to enable DaphneDSL's `registerView()`+`sql()`-style of executing SQL queries in DaphneLib.
2. If we want to calculate multiple results, calling `compute()` for each of them would be quite inefficient. Especially if they are generated by overlapping DAGs of operations, we would repeat a lot of the computations. There should be a way to generate the merged script for multiple results at once.
Contributor guide
Research direction
Start at DaphneLib/Python's compute() entry point and trace how a DAG node becomes a DaphneDSL script. Compare result-producing nodes with side-effect operations such as print(), write(), and registerView(), and examine how registerView()+sql() should work. Done means one execution path can emit a merged script for multiple results while preserving side effects and shared computations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100