rivet-dev / rivet-dev/agentos

Native root filesystem returns stale data after a guest process overwrites an API-created file

Open
#1,901 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
4.6k
Forks
251
Avg merge
2d 15h
Merged PRs (30d)
34

Description

Summary

With a sidecar-native chunked_actor_sqlite root filesystem, a file created through vm.filesystem.writeFile() is not updated in the filesystem API view after /bin/sh overwrites it.

The shell sees the new content, while vm.filesystem.readFile() deterministically returns the old content. Disposing and reopening the VM with the same SQLite database also returns the old content, so the guest update is not persisted to the authoritative VFS.

Minimal reproduction

import { mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { AgentOs } from "@rivet-dev/agentos";

const dir = await mkdtemp(join(tmpdir(), "agentos-fs-repro-"));
const sqlitePath = join(dir, "workspace.sqlite");
const sidecar = await AgentOs.createSidecar({ placement: "process" });
const vm = await AgentOs.create({
  sidecar: { kind: "explicit", handle: sidecar },
  database: { type: "sqlite_file", path: sqlitePath },
  rootFilesystem: {
    type: "native",
    plugin: {
      id: "chunked_actor_sqlite",
      config: { namespace: "root", uid: 1000, gid: 1000 },
    },
  },
  permissions: {
    fs: "allow",
    network: "deny",
    childProcess: "allow",
    process: "allow",
    env: "allow",
    binding: "deny",
  },
});

try {
  await vm.filesystem.mkdir("/workspace", { recursive: true });
  await vm.filesystem.writeFile("/workspace/shared.txt", "before");

  const proc = await vm.process.spawn(
    "/bin/sh",
    ["-c", "printf after > shared.txt; cat shared.txt"],
    { cwd: "/workspace", output: { retainEvents: true } },
  );
  await vm.process.wait(proc.pid);

  const events = await vm.process.getOutput(proc.pid);
  console.log(events); // shell output contains "after"
  console.log(
    Buffer.from(await vm.filesystem.readFile("/workspace/shared.txt")).toString(),
  ); // actual: "before", expected: "after"
} finally {
  await vm.dispose();
  await rm(dir, { recursive: true, force: true });
}

Equivalent minimal sequence:

filesystem.writeFile("/workspace/shared.txt", "before")
process.spawn("/bin/sh", ["-c", "printf after > shared.txt; cat shared.txt"])
filesystem.readFile("/workspace/shared.txt")

shell stdout: after
filesystem API: before

Reproduction matrix

Sequence Result
API write -> API read Correct
Shell create -> API read Correct
Shell create -> Shell overwrite -> API read Correct
API write -> Shell overwrite -> API read Stale pre-overwrite content
API write -> Shell remove/recreate -> API read ENOENT / not found

The failure remains when:

  • the replacement has a different byte length;
  • waiting 100 ms before the overwrite;
  • waiting 100 ms and touching the file after the overwrite;
  • disposing and reopening the VM against the same SQLite file.

If the initial file is also created through a guest process (cat > file via stdin), subsequent guest overwrites are visible through filesystem.readFile().

Expected behavior

Host filesystem APIs and guest processes should observe the same VFS. After the process exits successfully, filesystem.readFile() should return after, and reopening the persisted VM should retain after.

This matches the filesystem architecture documentation:

Host-side APIs (agent.writeFile, agent.readFile) enter the same VFS from the trusted side.

Suspected boundary

The behavior appears specific to native sidecar shadow-root reconciliation:

  • API WriteFile updates the kernel VFS and mirrors it into the process shadow root.
  • /bin/sh updates the shadow-root file successfully.
  • The process-exit shadow -> kernel reconciliation does not update the API-created inode in the authoritative VFS.

Likely relevant paths:

  • crates/native-sidecar/src/filesystem.rs
    • mirror_guest_filesystem_shadow_after_call
  • crates/native-sidecar/src/execution/launch.rs
    • sync_process_host_writes_to_kernel
    • sync_vm_shadow_root_to_kernel
    • sync_host_directory_tree_to_kernel_inner
  • crates/native-sidecar/src/plugins/chunked_actor_sqlite.rs

The existing vfs-consistency.nightly.test.ts exercises the runtime test kernel, but does not appear to cover the combination of a native process shadow and chunked_actor_sqlite after an API-created file is overwritten.

Environment

  • @rivet-dev/agentos: reproduced on 0.2.16-rc.1 and 0.2.16-rc.2
  • Node.js: v22.22.2
  • OS: Linux x86_64, kernel 6.6.98
  • glibc: 2.38
  • Root plugin: chunked_actor_sqlite
  • Reproduction rate: deterministic across repeated runs

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with crates/native-sidecar/src/filesystem.rs and the sync functions in crates/native-sidecar/src/execution/launch.rs, then inspect the chunked_actor_sqlite plugin. Run the existing vfs-consistency.nightly.test.ts and extend coverage for an API-created file overwritten by a native guest process. Done means filesystem reads and reopening the SQLite-backed VM both return the guest's updated content.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sqlite, typescript
Domain
databases, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.