haskell-servant / haskell-servant/servant
Make `fromAction` and `fromActionStep` able to pass a value from an action to the next one
- Dominant language
- Haskell
- Stars
- 2k
- Forks
- 427
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 5
Description
## Context
On `servant-server`, I must serve some data that I query from a store. The persistance interface returns me some [`Stream`](https://hackage.haskell.org/package/streaming-0.2.3.0/docs/Streaming-Prelude.html) of resources, so actual queries to the store are done only when the stream is consumed.
### Unsatisfactory solution
Fold the stream into a list and respond with that list in a normal `Servant` endpoint. This is unsatisfactory because by folding the stream, all resources are read from the persistance layer and accumulated in memory before being sent, which leads me to huge memory usage.
## Objective
Pipe the resources directly from the `Prelude.Streaming.Stream` into a `Servant.Types.SourceT`, so that the persistance layer is queried only when the resource must be sent in the response stream, and thus avoid to accumulate many resources in memory.
To achieve this, I can generate a `SourceIO` with a [`fromAction`](https://hackage.haskell.org/package/servant-0.18.3/docs/Servant-Types-SourceT.html#v:fromAction) or `fromActionStep`, that reads the next element of the stream.
## Issue
Since all the actions run by `fromAction` are actually the same, I cannot consume the stream by accessing the next values.
## Solution
Tweak `fromActionStep` to make it pass a value from an action to the next one:
```haskell
fromActionStep' :: Functor m => (c -> m (Maybe (a,c))) -> c -> StepT m a
fromActionStep' action = loop where
loop c = Effect $ step <$> action c
step Nothing = Stop
step (Just (x,t)) = Yield x $ loop t
```
And then call it on `Prelude.Streaming.uncons`.
Of course, `Maybe (a,c)` can be generalized to any type `v`, provided that we pass additionally some `stop :: v -> Bool`, `value :: v -> a` and `rest :: v -> v` functions.
## Question
Was there an easier out-of-the-box solution I did not see?
Otherwise, is it worth adding such `fromActionStepWithConsumable` method to the library?
Contributor guide
Assessment
This issue has not been assessed yet.