Add support for multi-tenancy
- Dominant language
- Python
- Stars
- 183
- Forks
- 41
- Avg merge
- 8h 6m
- Merged PRs (30d)
- 2
Description
## Overview
Loom currently assumes a single-tenant environment: one database, one Cognito/OIDC identity provider, and a single flat namespace of agents, memories, settings, and other resources shared by every user of a deployment. SaaS providers that want to expose Loom to their own customers need multi-tenant support instead — separate tenant environments, with each tenant's agentic resources and data strictly isolated from every other tenant's.
This issue adds tenant as a first-class concept: administrators can create and manage tenants, every tenant-scoped resource is filtered by tenant on every request, and tenants get limited self-service customization of their own environment.
## Context
### Current State
**No tenant concept exists anywhere in the data model**
- `backend/app/models/__init__.py` L1-27 lists every ORM model — `Agent`, `InvocationSession`, `Invocation`, `ConfigEntry`, `CredentialProvider`, `Integration`, `ManagedRole`, `AuthorizerConfig`, `PermissionRequest`, `AuthorizerCredential`, `Memory`, `TagPolicy`, `TagProfile`, `McpServer`/`McpTool`/`McpServerAccess`, `SiteSetting`, `AuditLogin`/`AuditAction`/`AuditPageView`, `ApprovalPolicy`, `ApprovalLog`, `VpcConfig` — none has a `tenant_id`/`organization_id` column.
- There is **no persisted `User` model at all**. Identity is resolved fresh on every request from JWT claims into an in-memory `UserInfo` dataclass (`backend/app/dependencies/auth.py` L129-149: `sub`, `username`, `groups`, `scopes`, `idp_type`) and never written to the database.
- Every `grep` hit for `account_id` (`backend/app/models/agent.py` L31, `backend/app/models/memory.py` L24, and usage throughout `backend/app/routers/agents.py`, `backend/app/routers/memories.py`, `backend/app/services/iam.py`) refers to an **AWS account ID** used for building ARNs — unrelated to tenancy.
- `Agent` (`backend/app/models/agent.py` L18-45) is a flat table: `id`, `arn`, `runtime_id`, `name`, `status`, `region`, `account_id`, plus deployment/config metadata — no owner or tenant scoping column.
**Auth already resolves identity per-request from OIDC claims — the natural place to thread a tenant claim through**
- `get_current_user()` (`backend/app/dependencies/auth.py` L229-284) validates the `Authorization: Bearer` header fresh on every request; nothing about identity is cached server-side between requests.
- Two auth paths exist: external OIDC IdP (Entra ID / Okta / Auth0 / generic OIDC), configured per `IdentityProvider` row (`backend/app/models/identity_provider.py` L7-27), or Cognito fallback (`auth.py` L284-292).
- `_build_user_from_external_claims` (`auth.py` L300-328) already maps arbitrary external-IdP claims into internal Loom groups (`t-admin`/`t-user`/`g-*`) via a per-IdP `group_mappings` JSON column (`identity_provider.py` L19) — the same mechanism could map a `tenant_id`/`org_id` claim into `UserInfo`, but nothing downstream (routers, models, queries) consumes such a value today.
- The recent fail-closed fix (`auth.py` L15-30) means an IdP must be configured for auth to succeed at all outside loopback local-dev — there is no silent bypass to account for when adding tenant checks.
**Settings are a single global key/value table, not scoped to anything**
- `backend/app/models/site_setting.py` L8-14: `SiteSetting.key` is globally `unique=True` with a single `value` column.
- `get_site_setting()` (`backend/app/routers/settings.py` L34-38) does `db.query(SiteSetting).filter(SiteSetting.key == key).first()` with no scoping filter — `enabled_model_ids`, `loom_registry_id`, `litellm_proxy_base_url`, etc. (`SITE_SETTING_DEFAULTS`, `settings.py` L26-31) apply to the entire deployment, not per tenant.
**No scoping mechanism exists anywhere in the query layer**
- `get_db()` (`backend/app/db.py` L47-56) yields a plain `SessionLocal()` bound to one global `engine` and closes it after the request — no per-request scoping context, no query-filtering hook, no row-level security.
- Every router (`agents.py`, `memories.py`, `settings.py`, etc.) issues plain `db.query(Model)...` calls with no predicate beyond the resource's own filters — there is no shared base-query helper to inject a tenant filter into, and most models have no tenant column to filter on regardless.
- `_migrate_add_columns()` (`backend/app/db.py` L60-152) is the existing "ALTER TABLE to add missing columns" migration mechanism (Postgres vs. SQLite branching already handled) — this is the natural place to land a `tenant_id` backfill migration.
**Infra is single-stack, single-DB, single-IdP-pool per deployment**
- `backend/iac/rds.yaml` L195-202: one `AWS::RDS::DBInstance` (Postgres 15) per deployment — no per-tenant database or schema provisioning today.
- `shared/iac/cognito.yaml` L34-35: one `AWS::Cognito::UserPool` per deployment.
- Today, "environments" means whole separate CloudFormation stack deployments (one DB, one Cognito pool, one ECS service each) — there is no notion of multiple tenants living inside a single running stack. This issue assumes schema-level isolation (single stack, `tenant_id` columns + query filtering) rather than provisioning a new stack per tenant, since the latter is a much larger infra undertaking; R1 should make this an explicit, documented design decision rather than an implicit assumption.
**Frontend has a theming mechanism to build on, but it's per-browser, not per-tenant**
- `frontend/src/contexts/ThemeContext.tsx` L3-18/L47: 10 named color themes stored in `localStorage` under `loom-theme` — client-side only, not server-persisted, not admin-configurable.
- Logo assets are static files in `frontend/public/assets/` (`loom_dark.png`, `loom_light.png`, etc.), selected by light/dark mode in `frontend/src/App.tsx` L516 — no config-driven or per-tenant override exists.
### Key Files
- `backend/app/models/__init__.py`, and every model under `backend/app/models/` — the full set of tables that need a `tenant_id` column and query-side scoping
- `backend/app/dependencies/auth.py` — `UserInfo`, `get_current_user`, `_build_user_from_external_claims` — where a tenant claim needs to be resolved and attached to the authenticated identity
- `backend/app/models/identity_provider.py`, `backend/app/routers/identity_providers.py` — per-IdP claim mapping, the likely home for a tenant-claim mapping config
- `backend/app/db.py` — `get_db()`, `_migrate_add_columns()` — where request-scoped tenant filtering and the schema migration for new `tenant_id` columns need to land
- `backend/app/models/site_setting.py`, `backend/app/routers/settings.py` — global settings table that needs a tenant-scoped counterpart for R4
- `frontend/src/contexts/ThemeContext.tsx`, `frontend/src/App.tsx` L516, `frontend/public/assets/` — existing theming/branding touchpoints to extend for tenant customization
## Requirements
### R1: Administrator can add and administer separate tenant environments
- Add a `Tenant` model (id, name, status, creation metadata) and an admin-only CRUD surface (list/create/update/deactivate tenants), following the existing router/model conventions (e.g. `backend/app/routers/admin.py`, `backend/app/routers/identity_providers.py`)
- Extend `IdentityProvider` (or its `group_mappings` mechanism, `identity_provider.py` L19) so a tenant can be resolved from an OIDC claim (or group) at login time, and thread the resolved tenant onto `UserInfo` (`auth.py` L129-149) alongside `sub`/`groups`/`scopes`
- Explicitly decide and document the isolation model this issue targets — schema-level (single stack/DB, `tenant_id` columns + query filtering) vs. stack-level (separate CloudFormation stack per tenant) — since today's infra (`backend/iac/rds.yaml`, `shared/iac/cognito.yaml`) is single-stack and the rest of this issue assumes schema-level isolation
- Administrators need a way to act *as* or *for* a given tenant (e.g. an admin-only tenant-switcher) to administer tenant-specific resources without needing separate admin credentials per tenant
### R2: Tenants cannot see any agentic resources from other tenants
- Add a `tenant_id` column to every tenant-scoped model — at minimum `Agent`, `InvocationSession`, `Invocation`, `Memory`, `ConfigEntry`, `CredentialProvider`, `Integration`, `ManagedRole`, `AuthorizerConfig`, `McpServer`/`McpTool`/`McpServerAccess`, `ApprovalPolicy`, `ApprovalLog` — via the existing `_migrate_add_columns()` pattern (`backend/app/db.py` L60-152)
- Every router that queries these models (`agents.py`, `memories.py`, `mcp.py`, `security.py`, `approvals.py`, etc.) must filter by the requesting user's `tenant_id` on every read and write — since there's no shared base-query helper today, this needs either a new query-filtering dependency/helper introduced across all routers, or a per-model scoping mixin, to avoid re-implementing the filter ad hoc (and inconsistently) in every endpoint
- `SiteSetting`-driven behavior that is meant to be global (e.g. `enabled_model_ids`) vs. tenant-specific needs to be explicitly categorized — see R4
- Add authorization tests confirming a user authenticated for tenant A gets 403/404 (not silently empty results) when directly requesting tenant B's resource by ID
### R3: Data cannot accidentally split over to other environments
- Given there is no existing base-query helper or row-level filtering (`backend/app/db.py` L47-56), evaluate Postgres row-level security (RLS) policies keyed off a per-request session variable as a defense-in-depth backstop, in addition to (not instead of) explicit `tenant_id` filtering in application code — a single missed filter in a new endpoint should not be sufficient to leak data
- Ensure `tenant_id` is set at creation time for every new row (agents, memories, sessions, etc.) from the authenticated request's resolved tenant, never from client-supplied input, to prevent a malicious or buggy client from writing into another tenant's namespace
- Add an integration/regression test suite that creates resources under two distinct tenants and asserts zero cross-tenant leakage across every tenant-scoped list/get endpoint — this should run in CI and fail the build on any new endpoint that omits tenant filtering
- Audit background/async jobs and shared caches (if any exist outside per-request DB sessions) for tenant leakage, since scoping only at the FastAPI dependency layer would miss anything running outside that request lifecycle
### R4: Tenants should have some ability to customize their own environments, even from a visualization standpoint
- Add a tenant-scoped settings store — either a `tenant_id` column added to `SiteSetting` (changing its unique constraint from `key` to `(tenant_id, key)`, `site_setting.py` L8-14) or a new parallel `TenantSetting` table — distinct from the global settings that must remain deployment-wide (e.g. `loom_registry_id`)
- Extend the existing theming mechanism (`frontend/src/contexts/ThemeContext.tsx`) so a tenant's chosen theme/branding is persisted server-side per tenant (not just `localStorage`) and applied automatically for that tenant's users, rather than left as a personal per-browser preference
- Support tenant-level logo/branding override (replacing the static bundled assets in `frontend/public/assets/`, referenced in `frontend/src/App.tsx` L516) via a config-driven or uploaded-asset mechanism
- Scope this to visual/branding customization only for this issue (theme, logo, display name) — functional customization (e.g. per-tenant feature flags controlling agent capabilities) is a larger design question and out of scope here
## Testing
- Add unit tests for the `Tenant` model and admin CRUD endpoints
- Add unit tests verifying `UserInfo`/`get_current_user` correctly resolves and attaches `tenant_id` from OIDC claims, including the case where no tenant claim is present (should fail closed, consistent with the existing fail-closed-on-missing-IdP behavior in `auth.py` L15-30 rather than defaulting to a shared/global tenant)
- Add the cross-tenant isolation regression suite described in R3 to CI (`cd backend && make test`)
- Add tests confirming tenant-scoped settings (R4) override global defaults correctly and that global-only settings remain unaffected by tenant scoping
- Manually verify: create two tenants, provision an agent under each, confirm each tenant's UI only shows its own agents, and confirm a tenant admin cannot access another tenant's resources by ID manipulation
## Out of Scope
- Stack-level tenant isolation (separate CloudFormation stack/DB/Cognito pool per tenant) — R1 requires documenting this as a considered alternative, but this issue implements schema-level isolation within a single deployment
- Per-tenant billing, usage metering, or quota enforcement — a future issue can build on the `tenant_id` scoping introduced here
- Functional (non-visual) per-tenant customization, such as tenant-specific feature flags or agent capability restrictions
- Self-service tenant signup/onboarding flows — tenant creation in this issue is administrator-driven only
Contributor guide
Research direction
Start by reading backend/app/dependencies/auth.py, backend/app/db.py, and the models and routers listed under R1-R4; the isolation model is an explicit design decision. Review the existing migration and authentication tests before mapping tenant coverage across backend and frontend entry points. Done means tenant CRUD, claim resolution, scoped data and branding, plus cross-tenant regression tests running in CI.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, fastapi, postgresql, python, typescript
- Domain
- authentication, authorization, backend, cloud, databases, frontend, security, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100