rivet-dev / rivet-dev/agentos

Question: expected sessionEvent throughput and session boot latency via the RivetKit actor (vs agentos-core)?

Open
#1,754 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
4.6k
Forks
251
Avg merge
2d 15h
Merged PRs (30d)
34

Description

We're building a platform on @rivet-dev/agentos 0.2.7 (RivetKit actor path, one agentOS() VM actor per agent session, following the quickstart/llm-credentials/react examples) and we're seeing much higher latencies through the actor than when driving the same workload via @rivet-dev/agentos-core directly. Before assuming anything is wrong upstream, we'd like to check whether we're holding it wrong, and understand the design intent.

Measurements

Host: Hetzner VPS, 2 vCPU / 4GB, Linux x64, node 22. Agent: pi, streaming from a fast provider (Cerebras, ~2,100 tok/s raw from the same host).

Metric via RivetKit actor via agentos-core directly
sessionEvent cadence during streaming ~2.7 events/s (p50 gap ~370ms) ~340 chunks/s (p50 gap 3ms, ~1,400 tok/s)
createSession("pi") 14–17s ~1s
single action round-trip (exists) ~60ms

One thing we stumbled on: @rivet-dev/agentos pins rivetkit to 0.0.0-stack-feat-rivetkit-forward-inspector-tabs-..., whose engine binary is ~208MB (the released 2.3.x engine is ~85MB). Overriding RIVET_ENGINE_BINARY with the released 2.3.2 engine improved things substantially (event cadence 2.7/s → ~21/s, boot 14–17s → ~8.4s), which makes us wonder if the pinned preview engine is an unoptimized build. Is running a released engine under the preview rivetkit JS supported, or are we off the paved road there?

What we think we're seeing (please correct us)

Reading crates/agentos-actor-plugin, it looks like:

  • spawn_event_capture awaits insert_session_event (a SELECT MAX(seq) + INSERT) inline in the event pump loop, so broadcast cadence is gated by two SQL round-trips per event (~25ms each via the host ABI on our box).
  • handle_fs_call runs ensure_fs_root (an INSERT OR IGNORE) before every guest fs operation, and each fs op is a synchronous round-trip, which seems to dominate VM boot (the durable FS tree is written row-by-row).

We also read the actor state docs recommending a memory working-copy with batched write-back, so we're curious about the trade-off chosen here.

Questions

  1. Are these latencies expected for the actor path on modest hardware, or does this smell like misuse on our side (actor-per-session granularity, config, missing option)?
  2. Is the synchronous per-event persistence a deliberate durability guarantee (e.g., "no event is ever broadcast that isn't already durable")? If so, is there interest in an opt-out or batched mode for streaming-heavy sessions? We'd guess a memory-held seq counter + batched inserts would help, but there may be crash-safety or memory-cost reasons (large FS trees, actor migration) that make that unattractive — we'd genuinely like to understand.
  3. Is RIVET_ENGINE_BINARY + released engine a reasonable production setup until the rivetkit pin moves to a release?

Happy to share repro scripts (they're short) or test patches. And separately: 0.2.8-rc.1/rc.2 currently don't install from npm — they depend on @rivet-dev/agentos-plugin@0.2.8-rc.x, which has no published versions. Wasn't sure if that deserves its own issue; can split it out if useful.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with crates/agentos-actor-plugin, especially spawn_event_capture, insert_session_event, handle_fs_call, and ensure_fs_root, then compare the actor and agentos-core paths using the reported measurements. Check the rivetkit pin and RIVET_ENGINE_BINARY behavior against the documented actor state model. Done should establish whether the latency is expected, whether the engine override is supported, and what follow-up is warranted.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, linux, nodejs, rust, wasm
Domain
backend, databases, distributed-systems, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.