ash-project / ash-project/ash

Synthesizing Aggregates across data layer boundaries

Open
#560 0 comments 0 reactions 0 assignees View on GitHub
advanced enhancement
Dominant language
Elixir
Stars
2.5k
Forks
422
Avg merge
23h 26m
Merged PRs (30d)
46

Description

Currently the ETS data layer has an implementation of aggregates that fetches the relevant records and counts/sums them up. What we can do at the framework level is implement that outside of the ETS data layer and instead do it as a synthesized operation in Ash core. What this means is that if a data layer can't aggregate, or if you were to do something like this:

```elixir
count :count_of_foos, :foos
```

where `:foos` is in a different data layer, we can fetch the relevant foos and count them up. We can leverage various data layer features to do this smartly. For example, if the data layer supports aggregating against queries, we could construct a query that would yield the appropriate result, and aggregate that in the other data layer for each record. Additionally, if you cross data layer boundaries multiple times with something like `:count` we could add anonymous aggregates at each level for the count at that relationship path, and then sum those up (wouldn't work with average, for example, that would need to be switched under the hood to get sum/count at each stage, and then turned into average afterwards).

i.e

```elixir
count :count_of_bars, [:foo, :bar]
```

Where `:foo` and `:bar` are in separate data layers. We could get the relevant foos, and load the anonymous aggregate equivalent of `count_of_bars`, and then sum those up to produce the count of bars for this resource.

Contributor guide

Open the contributing guide

Research direction

No files or tests are named. Start by comparing the existing ETS data-layer aggregate implementation with the proposed Ash core operation, then map how relationships cross data layers; done means aggregates such as counts can be synthesized across those boundaries, with the limitations described in the issue addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
elixir
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.