Optimize traverseWithChain for StackSafeMonad(s)
Nobody has claimed this yet.
- Dominant language
- Scala
- Stars
- 5.5k
- Forks
- 1.2k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 5
Description
The traverseWithChain implementation is really good and performs well in benchmarks on downstream datatypes, but it has a meaningful flaw: it forces added allocations through Eval at every step of the iteration. This is something that is definitely necessary when you're trying to implement it on an arbitrary applicative, but that really isn't the case most of the time.
The common scenario I'm thinking about is traversing List with IO, which happens both in the context of parTraverse and the more conventional traverse, and it happens very frequently. When you instantiate G = IO within traverseWithChain, the map2Eval and corresponding machinery is no longer necessary because IO itself is stack-safe, and thus we can entirely remove that wrapping allocation in each step.
My proposal would be to do this by checking to see if G.isInstanceOf[StackSafeMonad[G]]. The reasoning here is that most such implementations will take advantage of the StackSafeMonad utility mixin. For those that don't, the old Eval implementation will still work, but for those that do, we have the guarantee (based on their own tailRecM laws) that their flatMap is capable of suspending the laziness just as safely as Eval's. This covers the common case of IO as well as several other similar cases, and achieves a significant speed-up in downstream performance without sacrificing anything other than ugliness of implementation.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate the traverseWithChain implementation and the StackSafeMonad definitions, then read how Eval and map2Eval are used during traversal. Verify the existing downstream benchmarks or tests, and consider the issue complete when StackSafeMonad instances avoid the per-step Eval allocation while the general applicative fallback remains intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100