stacklok / stacklok/mecatl

Cloud-native: multi-tenant memory scope key (decision a)

Open
#61 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
152
Forks
16
Avg merge
14h 48m
Merged PRs (30d)
536

Description

Summary

Multi-tenant memory scope key. This tracks decision (a) in docs/design/CLOUD-NATIVE.md (List 3). Cross-session memory is opened from cfg.MemoryDir once at build time, with per-project-directory semantics, shared by every session in the process. Two things make this a real problem for a shared/cloud deployment:

  • A no-FS cloud session (issue #55) has no directory, so "which memory does this session see" has no obvious answer.
  • The MemoryStoreService driver protocol carries nothing scope-shaped: RecallRequest{key}, ListRequest{prefix}, IndexRequest{}. No tenant, principal, session, or namespace field anywhere. So today the scope boundary is the endpoint: whatever --memory-store-url points at is the scope.

Per-deployment (the status quo) is rejected: it does not allow resource-sharing (one deployment serving many tenants or agents with isolated memory). We want principal/tenant keying (option 2) and/or per-agent-identity keying (option 3).

This is not part of Phase 4 (leasing). It is an orthogonal multi-tenant concern that shares the cloud motivation but neither blocks nor depends on leasing.

Weighing option 2 vs option 3

They are orthogonal axes, not mutually exclusive: option 2 scopes by who (tenant/principal), option 3 by which agent. A mature deployment plausibly wants both, so tenant T's agent A has memory distinct from T's agent B and from tenant T2's agent A.

The decisive point that orders them: option 3 alone is a cross-tenant leak. If memory is keyed only on agent identity, one agent's memory is shared across every tenant using that agent. So in a multi-tenant context the tenant/principal dimension (2) is the required outer scope, and agent identity (3) is an optional inner sub-scope. The real key is tenant or tenant/agent, never agent alone.

The clean mechanism: an opaque memory scope key (a namespace string) on the MemoryStore port and the driver, computed by composition from a configurable policy. The store and the wire see an opaque namespace; composition decides whether that string is the tenant, the agent, or tenant/agent. This matches the repo's opaque-label discipline (ProviderID, Workspace: the domain stores them, never interprets them), supports option 2, option 3, and composite, and keeps a keying policy out of the protocol.

The dependency that sequences this

mecated's auth today is a single shared bearer token (internal/adapter/server/authn.go; an empty token disables auth for dev). There is no per-principal or per-tenant identity model, and CreateSession carries no caller identity. Consequence:

  • Option 3 (agent identity) is implementable now. The agent/soul is selected per session, and per-agent persistent memory (issue #33) already keys a read-only MEMORY.md on the agent def name under an FS root. This is the read-path precedent to generalize to the write path.
  • Option 2 (tenant/principal) is gated on a per-principal auth/identity model that does not exist yet (multiple tokens mapped to principals, or a per-connection principal). That model is the real prerequisite.

Proposed sequencing

  1. Build the opaque scope-key mechanism on the MemoryStore port and the MemoryStoreService driver now (additive, forward-looking).
  2. Land option 3 (agent sub-scope), generalizing #33's read-path keying to the six memory tools' write path. Implementable today.
  3. Design the per-principal/tenant identity model as a prerequisite (linked issue) for option 2.
  4. Compose tenant as the outer scope when identity arrives, with agent as the optional inner sub-scope. Never agent-alone in a multi-tenant context.

Open questions

  • Scope-key shape: a single opaque string vs a structured (tenant, agent) value. (Leaning opaque string, composed by the policy.)
  • Where composition derives each dimension: the auth principal for tenant, the soul/agent def for agent.
  • The no-FS session's memory scope specifically (it has no directory to fall back to).
  • Migration of the existing per-project-directory store (does the dir become one policy among several?).
  • Whether #33's FS-rooted read-only per-agent memory unifies with this scope key or stays a separate read-only injection.

Acceptance criteria

  • An opaque scope key on the MemoryStore port and the MemoryStoreService driver (additive, no break to existing single-scope deployments).
  • A configurable composition policy that can produce a tenant, agent, or composite key.
  • Option 3 write-path scoping (the six memory tools honor the agent sub-scope).
  • The conformance suite covers scope isolation (entries under one key are invisible under another).
  • The per-principal identity prerequisite for option 2 is tracked (sub-issue or link).
  • A no-FS session resolves a well-defined memory scope.

Relationship

  • Decision (a) in docs/design/CLOUD-NATIVE.md.
  • Independent of Phase 4 (leasing); both are multi-tenant cloud prod requirements but neither blocks the other.
  • The per-principal auth/identity model is a prerequisite for the option-2 half and should be its own issue.

Non-goals

  • Building the auth/identity model itself (prerequisite, tracked separately).
  • Cross-tenant shared memory (explicitly isolated; shared org knowledge, if ever wanted, would be a deliberate separate scope).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with docs/design/CLOUD-NATIVE.md and internal/adapter/server/authn.go, then trace the MemoryStore port, driver, and six memory tools. Done means the additive opaque scope key and configurable composition are implemented, agent-scoped writes and no-FS behavior are defined, and the conformance suite verifies isolation without breaking single-scope deployments.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
api, backend, cloud, distributed-systems, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.