typelevel / typelevel/cats-effect

Can we push to make `IOFiber` fit in 64 bytes?

Open
#3,584 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

:microscope: experiment
Dominant language
Scala
Stars
2.2k
Forks
576
Avg merge
2d 11h
Merged PRs (30d)
18

Description

As of Cats Effect 3.5.0-RC5, this is the memory layout of IOFiber:

cats.effect.IOFiber object internals:
OFF  SZ                                TYPE DESCRIPTION               VALUE
  0   8                                     (object header: mark)     N/A
  8   4                                     (object header: class)    N/A
 12   4                                 int AtomicBoolean.value       N/A
 16   4                                 int IOFiber.masks             N/A
 20   1                                byte IOFiber.resumeTag         N/A
 21   1                             boolean IOFiber.canceled          N/A
 22   1                             boolean IOFiber.finalizing        N/A
 23   1                                     (alignment/padding gap)   
 24   4      scala.collection.immutable.Map IOFiber.localState        N/A
 28   4   scala.concurrent.ExecutionContext IOFiber.currentCtx        N/A
 32   4              cats.effect.ArrayStack IOFiber.objectState       N/A
 36   4              cats.effect.ArrayStack IOFiber.finalizers        N/A
 40   4           cats.effect.CallbackStack IOFiber.callbacks         N/A
 44   4                    java.lang.Object IOFiber.resumeIO          N/A
 48   4        cats.effect.unsafe.IORuntime IOFiber.runtime           N/A
 52   4      cats.effect.tracing.RingBuffer IOFiber.tracingEvents     N/A
 56   4                               int[] IOFiber.conts             N/A
 60   4          cats.effect.kernel.Outcome IOFiber.outcome           N/A
 64   4                      cats.effect.IO IOFiber._cancel           N/A
 68   4                      cats.effect.IO IOFiber._join             N/A
Instance size: 72 bytes
Space losses: 1 bytes internal + 0 bytes external = 1 bytes total

This is already amazing, considering that we were pushing well over 100 bytes around CE 3.3. Does it even make sense to try to get it any lower? My idea is that some x86 CPUs still have 64 byte cache line sizes. This is probably not true for ARM silicon.

Some ideas on the data structures. localState seems very rarely used, IMO, it doesn't need to be a direct field. Could it hold a permanent place as the 0th index of objectState? Could this trick also be used to store other data? On the surface, the only problem I can foresee is that it would force the backing array of objectState to be initialized earlier than it is now. Now, it is initialized when we first run the fiber, not when it is allocated. That could be a problem.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by inspecting the IOFiber layout and measuring its allocation and cache-line impact on the supported runtimes. Evaluate the proposed localState/objectState trade-off and any alternatives, then document a concrete design and benchmarks showing whether a 64-byte target is worthwhile.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
backend, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.