hoangsonww / hoangsonww/ReproVM-Virtual-Machine

Feature: Remote CAS (HTTP/S3) with Read-Through Cache & Lazy Fetch

Open
#2 0 comments 0 reactions 1 assignee Claimed by @hoangsonww View on GitHub
bug documentation enhancement good first issue help wanted question
Dominant language
C
Stars
11
Forks
7
PR merge metrics
No merged PRs in 30d

Description

**Summary**
Add an optional **remote Content-Addressed Storage backend** so multiple machines can share artifacts. Local `.reprovm/cas` stays as the primary store; when a blob is missing it’s **lazily fetched** from a configured remote (HTTP(S) or S3). Writes push new blobs upstream (configurable). This enables cache sharing across dev boxes and CI while keeping the current local semantics.

**Why**

* Massive speedups on CI and multi-host workflows (reuse prior artifacts).
* Deterministic, content-addressed reuse without changing the manifest format.
* Still works offline via local cache; remote is best-effort.

---

## Scope (MVP)

### New config (defaults shown)

```ini
# .reprovmrc (or env vars)
[remote_cas]
enable = false
mode = "read_through" # read_through | read_write
endpoint = "https://cas.example.com/objects" # or "s3://my-bucket/objects"
concurrency = 4
connect_timeout_ms = 2000
read_timeout_ms = 15000
retries = 2
backoff_ms = 250

[auth] # used when endpoint requires auth
aws_profile = "" # for S3 (SigV4)
aws_region = ""
bearer_token = "" # for HTTPS bearer
```

### CLI flags (override config)

* `--remote-cas=` (enables remote, sets endpoint)
* `--remote-mode=read_through|read_write`
* `--remote-concurrency=N`

### Behavior

* **Read path**: On missing local blob `objects/aa/bb...`, try remote `GET /aa/bb...`

* On 200: stream to temp, verify SHA-256, move into local CAS, serve.
* On 404/timeout: behave as today (blob remains missing).
* **Write path** (only in `read_write`): After local store succeeds, **async push** `PUT`/S3-upload; failures don’t fail the task but are logged.
* **Integrity**: Always verify SHA-256 of fetched bytes; refuse mismatches.
* **Concurrency**: Prefetch in parallel for tasks with many outputs (cap = `concurrency`).
* **Offline-friendly**: If remote is down, continue with local cache only.
* **No manifest changes required.**

---

## Acceptance Criteria

* With two hosts sharing the same remote CAS, a task built on Host A is **skipped** on Host B after lazy fetch (no command re-exec).
* If the remote is unreachable, runs still complete using local hits; only missing blobs fail as today.
* Upload of new blobs works in `read_write` and is skipped in `read_through`.
* SHA-256 verification happens on every download; mismatches are rejected and logged.
* Unit tests cover: HTTP 200/404/5xx, retry/backoff, checksum mismatch, and S3 happy-path (guarded / mocked).
* Docs show examples for HTTP and S3, and how to secure endpoints.

---

## Out of Scope (v1)

* Auth key rotation UX, multipart S3 uploads for >5GB, encryption-at-rest, delta sync, bloom-filter negotiation.
* Remote **metadata** sharing (we only sync blobs; `.meta` remains local).

---

## Implementation Sketch

**Interfaces (new):**

```c
// cas_remote.h
bool cas_remote_enabled(void);
bool cas_remote_fetch(const char *hex_hash); // downloads and stores into local CAS
void cas_remote_async_push(const char *hex_hash); // fire-and-forget in read_write
```

**Touch points:**

* In `cas.c` `cas_get_blob(...)`: if local miss and remote enabled → `cas_remote_fetch()`.
* After successful local store in build path: if mode is read\_write → queue `cas_remote_async_push()`.

**HTTP layout:**
Remote path mirrors local: `/objects/aa/bb...` (two-level hex sharding).

* GET for fetch; PUT for upload (or POST if server requires).
* Bearer token header if provided.

**S3 layout:**
Use bucket prefix `objects/aa/bb...` with SigV4 (env/profile).

* GET Object for fetch; PUT Object for upload.

**Concurrency:**
Single worker pool for fetch/push (bounded by `concurrency`). Serialize stdout to avoid interleaved TTY output.

**Retries:**
Exponential backoff: `backoff_ms * 2^attempt` (cap 2–3 tries).

**Security:**
TLS required for HTTPS; verify host/CA. Never trust remote checksum—always recompute locally.

---

## Docs & Examples

* Add `docs/remote-cas.md` with:

* Minimal HTTP server config (nginx static + write CGI, or simple Go file server)
* S3 bucket policy example and IAM permissions
* Example `.reprovmrc` and CLI overrides
* CI recipe: persist `.reprovm` locally and backfill from remote

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.