Epic: Implement OAuth2/OIDC Provider for First-Party SSO

## Summary

Make Backend.AI a standards-compliant **OAuth2/OIDC Provider** by building the identity layer in the **Account Manager** service — so that first-party applications and future third-party integrations can authenticate using standard protocols, enabling cross-app SSO, scoped tokens, token refresh/revocation, and external IdP federation.

## Background

### Current Architecture

- Manager issues **permanent keypairs** (`access_key`/`secret_key`) via `/auth/authorize`
- Each client stores these keypairs and re-signs every API request with HMAC-SHA256
- No shared session — users must log in separately to each app
- External SSO is handled by hook plugins that produce short-lived `sToken` JWTs via a non-standard flow
- No token expiry, no scoped access, no per-app revocation

**Manager's existing auth methods** (in `api/rest/middleware/auth.py`):
1. **HMAC-SHA256 signature** — `Authorization: BackendAI signMethod=HMAC-SHA256,credential=:` — two-level HMAC derivation with date+host, ±15 min clock skew tolerance
2. **JWT (HS256, per-user secret)** — `X-BackendAI-Token` header — signed with the user's own `secret_key`, used for GraphQL Federation (Apollo Router)
3. **Hook-based** — `PRE_AUTH_MIDDLEWARE` plugin dispatch — allows plugins (OpenID, custom-auth) to return an `access_key` for custom auth flows

**Existing SSO plugins** (in `manager/plugin/`):
- **OpenID plugin** (`plugin/openid/hook.py`) — listens on `AUTHORIZE` hook, reads `sToken` cookie, decodes HS256 JWT with a plugin-level shared secret, returns user row. Not standards-compliant OIDC — just a proprietary cookie-based JWT relay.
- **TOTP plugin** (`plugin/totp/hook.py`) — listens on `POST_AUTHORIZE` hook, validates TOTP codes via `pyotp`. Returns challenge responses (`REQUIRE_TWO_FACTOR_AUTH`, `REQUIRE_TWO_FACTOR_REGISTRATION`).

**Webserver's existing SSO proxy routes** (in `web/server.py:882-888`):
- Anonymous proxy routes already exist for SSO-related paths:
- `GET/POST /func/openid/*` → Manager OpenID plugin
- `POST /func/saml/*` → Manager SAML plugin
- `POST /func/custom-auth/*` → Manager custom auth plugin
- Token login handler (`token_login_handler`) accepts `sToken` cookie/body and forwards to Manager's `authorize()` hook

### Current Integration Pain Points (Investigated)

Concrete problems observed in **backend.ai-fasttrack** (third-party integration service) that motivate this work:

| Pain Point | Details | File Reference |
|---|---|---|
| **Bearer→HMAC conversion overhead** | FastTrack maintains its own Bearer token system (`UserToken` model, 40-char hex). Every `/func/*` proxy request must strip the Bearer token and re-sign with HMAC using stored `access_key`/`secret_key`. | `fasttrack/workflow/proxy/views.py:40-117` |
| **Duplicate user database** | FastTrack syncs user records from Manager to a local Django `User` model (storing `access_key`, `secret_key` locally). User data drifts if not re-synced. | `fasttrack/workflow/users/models.py:24-35`, `api/views.py:62-138` |
| **Epoch date HMAC workaround** | FastTrack uses a fixed epoch timestamp (0) for all HMAC signatures instead of real dates, working around date header sensitivity. References [#5737](https://github.com/lablup/backend.ai/issues/5737). | `fasttrack/workflow/common/auth/__init__.py:52` |
| **Session conflicts (409)** | Manager enforces `max_concurrent_logins` per user. When same user is logged into WebUI + FastTrack, Manager returns HTTP 409. FastTrack catches this and tells user to log out of WebUI first. | `fasttrack/workflow/users/api/views.py:299-314` |
| **Dual brute-force protection** | FastTrack implements its own per-IP rate limiting + per-username failure counting in Redis, on top of Manager's own login blocking. Duplicated security logic. | `fasttrack/workflow/users/api/bruteforce.py:1-149` |
| **2FA format fragility** | Manager returns 2FA requirements in two formats (old and new). FastTrack uses heuristic detection to handle both, with fallback for older Manager versions. | `fasttrack/workflow/users/api/views.py:328-352` |

**WebUI client** (backend.ai-webui-dev) also demonstrates the pattern each client must implement:
- Separate SESSION vs API connection modes (`react/src/helper/loginSessionAuth.ts:15-34`)
- Own session extension logic (`LoginSessionExtendButton.tsx`)
- Own credential storage/clearing in localStorage
- Own SSO integration per vendor (SAML form submission, OpenID form submission — `loginSessionAuth.ts:247-276`)

**Core problem**: Every new service must re-implement auth from scratch — login UI, credential storage, session management, HMAC signing, 2FA handling, brute-force protection. OAuth2/OIDC eliminates this entirely.

### Account Manager Service (Existing)

A **standalone microservice** (`src/ai/backend/account_manager/`) already exists with:
- Its own PostgreSQL database and Alembic migrations
- User model (`users`, `user_profiles` with role/status/password)
- Keypair model (`keypairs` with access_key, secret_key, expired_at)
- Application model (`applications` with `redirect_to`, `token_secret` — proto-OAuth client)
- User-to-application association (`association_applications_users`)
- Stubbed `/auth` and `/application` API sub-apps (empty, ready to implement)
- Separate `service-addr` (public: 8088) and `internal-addr` (internal: 8087) ports
- aiohttp-based, async, with plugin support

This is the **natural home** for OAuth2/OIDC — it was designed as a dedicated identity service.

### Existing Code Infrastructure to Leverage

Components already in the codebase that the OAuth2/OIDC implementation can build upon:

| Component | Location | What it provides | Reuse in SSO |
|---|---|---|---|
| **JWT library** | `common/jwt/` — `JWTSigner`, `JWTValidator`, `JWTConfig`, `JWTClaims` | Token generation/validation, claim types, exception hierarchy (`JWTExpiredError`, `JWTInvalidSignatureError`, etc.) | Extend for RS256 (currently HS256 only). Reuse `JWTClaims` structure and exception types. |
| **HMAC signature utils** | `common/auth/utils.py` — `generate_signature()` | HMAC-SHA signature generation for API requests | Keep for legacy auth. FastTrack already imports this (`from ai.backend.common.auth.utils`). |
| **Auth types & DTOs** | `common/dto/manager/auth/types.py` — `AuthTokenType`, `AuthResponseType`, `TwoFactorType`, `AuthSuccessResponse` | Polymorphic auth response models with discriminator pattern | Extend `AuthTokenType` with `OAUTH2_BEARER`. Add new response types for OAuth flows. |
| **RBAC permission types** | `common/data/permission/types.py` — `EntityType` (60+ types), `OperationType` (9 ops), `ScopeType`, `RBACElementType` | Fine-grained permission model already defines every entity and operation | Map OAuth scopes → `(EntityType, OperationType)` pairs for scope↔RBAC intersection. |
| **User context system** | `common/contexts/user.py` — `UserData`, `with_user()`, `current_user()` context managers | Async-safe user propagation via `contextvars` | OAuth2 Bearer auth populates the same `UserData` context as HMAC auth — downstream code unchanged. |
| **Password hashing** | Account Manager: `utils.py` — bcrypt (12 rounds). Manager: configurable algorithm with auto-migration. | Both components already hash passwords | Account Manager's password verification for the login UI in Phase 2. |
| **Hook plugin system** | `manager/plugin/` — `HookPlugin` base, `PRE_AUTH_MIDDLEWARE`/`AUTHORIZE`/`POST_AUTHORIZE` hooks | Extensible auth pipeline | Existing OpenID/TOTP plugins continue working during migration. New plugins could bridge to Account Manager. |
| **Valkey session client** | `common/redis/` — `ValkeySessionClient` | Redis/Valkey session storage with configurable TTL | Reuse for Account Manager's login session (Phase 2). |
| **Web proxy patterns** | `web/proxy.py` — `web_handler`, `anon_web_plugin_handler` | Authenticated and anonymous request proxying | Phase 1 uses same pattern to proxy `/oauth2/*` → Account Manager. |

### Target Architecture

```
┌──────────────────────────────────────────────────────────────┐
│ Account Manager (OIDC Provider) │
│ │
│ Public (service-addr :8088): │
│ /.well-known/openid-configuration │
│ /oauth2/authorize ← renders login UI │
│ /oauth2/token ← code exchange, refresh │
│ /oauth2/revoke ← token revocation │
│ /oauth2/userinfo ← user claims │
│ /oauth2/jwks ← public keys │
│ /oauth2/introspect ← token validation │
│ /oauth2/logout ← session invalidation │
│ /oauth2/device ← device auth verification page │
│ │
│ Internal (internal-addr :8087): │
│ /internal/users/* ← user CRUD (for Manager) │
│ /internal/keypairs/* ← keypair management (for Manager) │
│ /metrics ← Prometheus (already exists) │
│ │
│ DB: users, user_profiles, keypairs, applications, │
│ oauth2_tokens, oauth2_codes, oauth2_refresh_tokens │
│ │
│ Session: Redis/Valkey (login session for cross-app SSO) │
└──────────┬────────────────────────────┬──────────────────────┘
│ │
Public (OIDC) Internal API
│ │
┌──────▼──────┐ ┌─────────▼─────────┐
│ Webserver │ │ Manager │
│ (proxy) │ │ (compute API) │
│ │ │ │
│ Proxies │ │ Validates JWT │
│ /oauth2/* │ │ access tokens │
│ to Account │ │ via cached JWKS │
│ Manager │ │ │
└──────────────┘ └────────────────────┘
│ │
┌──────▼──────┐ ┌─────────▼─────────┐
│ App A │ │ App B │
│ (OIDC RP) │ │ (OIDC RP) │
└──────────────┘ └────────────────────┘
```

- **Account Manager** is the OIDC issuer — owns all OAuth2 endpoints, user identity, sessions, and token signing
- **Webserver** proxies `/oauth2/*` to Account Manager (like it proxies `/func/*` to Manager today) for deployments where Account Manager isn't directly browser-reachable
- **Manager** validates JWT access tokens via cached JWKS from Account Manager — no token signing responsibility
- **Clients** use standard Authorization Code Flow + PKCE against Account Manager (or Webserver as proxy)
- Legacy HMAC/keypair auth remains for backward compatibility (including CLI use)

## Key Design Decisions

| Decision | Answer | Rationale |
|---|---|---|
| **OIDC issuer** | Account Manager | Purpose-built identity service with existing user/keypair/application models. Webserver proxies for browser accessibility. |
| **Access token format** | JWT (RS256, 15 min TTL) | Offline validation via cached JWKS. Air-gapped friendly. Short TTL limits revocation window. |
| **Refresh token format** | Opaque (30 day TTL) | Stored in DB. Instantly revocable. Rotated on every use with reuse detection. |
| **Cross-app SSO** | Redirect-based only | Standard OIDC flow — works across any domain topology (cloud, on-prem, IP-only). |
| **Scope ↔ RBAC** | Intersection (`min(role, scope)`) | Scopes can narrow access, never widen. HMAC requests get implicit full scope. |
| **Domain in token** | Informational claim | Domain from user record, included in JWT `domain` claim. Enforcement stays in Manager's existing logic. |
| **Client registration** | Config-file seeded + Admin API | First-party clients in config, auto-seeded on startup. Third-party via API. |
| **CLI auth** | Keep legacy `/auth/authorize` | No ROPC (deprecated in OAuth 2.1). CLI uses existing endpoint or Device Authorization Grant. |
| **Token lifetimes** | Access: 15m, Refresh: 30d, Auth code: 60s | Per-client override in config. |

## Deployment Topology Considerations

### Cloud (SaaS)

```
account-manager.example.com ← OIDC issuer (or proxied via Webserver)
manager.example.com ← Compute API (validates JWT via JWKS)
webserver.example.com ← Proxies /oauth2/* to Account Manager
app-a.example.com ← OIDC Relying Party
app-b.example.com ← OIDC Relying Party
```

- HTTPS enforced at load balancer / CDN
- External IdP federation available (Google, Okta, etc.)

### On-Premise (Typical — different hosts)

```
10.0.0.1:8091 ← Manager (validates JWT via JWKS)
10.0.0.1:8088 ← Account Manager (co-located with Manager, internal)
10.0.0.2:8090 ← Webserver (proxies /oauth2/* to Account Manager)
10.0.0.3:9500 ← First-party app (browser-reachable)
```

- Account Manager may not be directly browser-reachable — Webserver proxies OAuth2 endpoints
- Cross-app SSO via redirect to Webserver → Account Manager (session exists → auto-approve)
- May run HTTP internally (TLS at corporate firewall)

### On-Premise (Single Node)

```
server.corp.local:8091 ← Manager
server.corp.local:8088 ← Account Manager
server.corp.local:8090 ← Webserver (proxies /oauth2/*)
server.corp.local:9500 ← First-party app
```

### Air-Gapped

- Zero external network access — no external IdP federation
- Local password auth + LDAP/AD as sole identity sources
- JWT access tokens validated offline via cached JWKS
- CLI uses legacy `/auth/authorize` endpoint

## Phases

### Phase 0: Quick Win — ID Token from Webserver (No Account Manager dependency)

**Motivation:** Before the full OIDC Provider is built, first-party apps need redirect-based login via the existing Webserver login page. This replaces the current pattern where each app implements its own login form and calls `/auth/authorize` directly.

**How it works:**

```
First-party app Webserver Manager
│ │ │
├─ GET /server/authorize? │ │
│ redirect_uri=& │ │
│ state= │ │
│ │ │
│ ┌──────────────────────────┤ │
│ │ Already has session? │ │
│ │ YES → skip to step 4 │ │
│ │ NO → show login page │ │
│ └──────────────────────────┤ │
│ │ │
│◀─ Webserver login page ──────┤ │
│ │ │
├─ User enters credentials ───▶│ │
│ ├─ POST /auth/authorize ───────▶│
│ │◀─ {access_key, secret_key} ──│
│ │ │
│ ├─ Store in session │
│ ├─ Sign ID Token (HS256 w/ │
│ │ shared secret, or RS256) │
│ │ │
│◀─ 302 redirect_uri? │ │
│ id_token=& │ │
│ state= │ │
│ │ │
├─ Validate id_token │ │
├─ Create local session │ │
├─ Use id_token claims for │ │
│ user identity │ │
```

**Webserver changes:**
- New endpoint: `GET /server/authorize` — accepts `redirect_uri` (allowlisted), `state`
- If session exists → sign ID Token → redirect back immediately
- If no session → render existing login page → after login → sign ID Token → redirect back
- ID Token (JWT) contains: `sub` (user UUID), `email`, `role`, `domain`, `access_key`, `exp` (short TTL, e.g. 5 min)
- Signing: HS256 with a shared secret (configured per first-party app), or RS256 if key infrastructure exists
- Allowlisted `redirect_uri` values in `config.toml` to prevent open redirect

**What this does NOT do:**
- No OAuth2 authorization code flow (ID Token returned directly via redirect — implicit-like, but first-party only)
- No scopes, no refresh tokens, no PKCE
- No client registration model — just an allowlist of redirect URIs
- Not standards-compliant OIDC — a pragmatic stepping stone

**What this enables:**
- First-party apps redirect to Webserver login page instead of implementing their own
- Cross-app SSO works — if user already has Webserver session, redirect returns ID Token immediately
- Apps get user identity (UUID, email, role) from a signed JWT instead of handling raw credentials
- **Migration path to full OIDC**: when Account Manager's `/oauth2/authorize` is ready, apps just change the redirect URL

**Acceptance criteria:**
- A first-party app can redirect to `/server/authorize` and receive a signed ID Token
- If Webserver session exists, redirect completes without showing login (SSO)
- ID Token is a valid JWT with user identity claims
- `redirect_uri` is validated against allowlist (no open redirect)
- Works with existing Webserver login page (no new UI needed)

---

### Phase 1: OAuth2/OIDC Provider in Account Manager

Build the OIDC Provider in the existing `account_manager` service — leveraging its user, keypair, and application models.

> **Important:** Phase 1 is **backend infrastructure only**. It implements OAuth2 endpoints, token signing, and Manager JWT validation — but has no session or production login UI. Apps should remain on Phase 0 for production use until Phase 2 adds sessions and cross-app SSO. Phase 1 is testable via programmatic OIDC flows (e.g., test scripts, Postman, conformance suite).

**Account Manager changes:**

- Extend `ApplicationRow` model to full OAuth2 client:
- Add: `client_id` (unique), `allowed_scopes`, `grant_types`, `app_type` (first_party/third_party), `auto_approve`, `access_token_ttl`, `refresh_token_ttl`
- Rename existing `redirect_to` → support multiple redirect URIs
- Existing `token_secret` → `client_secret`
- New models:
- `oauth2_access_tokens` (token hash, client_id, user_id, scopes, expires_at)
- `oauth2_refresh_tokens` (token hash, client_id, user_id, scopes, expires_at, rotated_at)
- `oauth2_authorization_codes` (code hash, client_id, user_id, redirect_uri, scopes, code_challenge, expires_at, used)
- Implement OAuth2 endpoints in `/auth` sub-app:
- `/.well-known/openid-configuration` — discovery document
- `/oauth2/authorize` — authorization endpoint (minimal test UI; production login UI in Phase 2)
- `/oauth2/token` — code exchange, refresh token rotation
- `/oauth2/revoke` — refresh token revocation
- `/oauth2/userinfo` — user claims from access token
- `/oauth2/jwks` — RS256 public keys
- `/oauth2/introspect` — token metadata for resource servers
- RS256 key pair generation, storage (etcd or DB), and JWKS rotation
- Config-file client seeding on startup
- Scope registry and validation
- PKCE required for all authorization_code grants
- JWT access token claims:
```json
{
"iss": "https://account-manager.example.com",
"sub": "",
"aud": "",
"exp": 1742919300,
"iat": 1742918400,
"scope": "openid profile pipelines:read",
"domain": "default",
"role": "admin",
"email": "user@example.com",
"name": "John Doe",
"kid": ""
}
```

**Manager changes:**

- Extend auth middleware (`api/rest/middleware/auth.py`) to accept `Authorization: Bearer `
- Fetch and cache JWKS from Account Manager's `/oauth2/jwks`
- Validate JWT signature, expiry, issuer, audience
- Extract user identity and scopes from claims
- Scope enforcement: `effective_permissions = role_permissions ∩ token_scopes`
- Existing HMAC/legacy JWT auth continues to work unchanged
- **Implementation note**: Manager currently has 3 auth methods tried in sequence (JWT via `X-BackendAI-Token`, HMAC via `Authorization: BackendAI ...`, Hook via `PRE_AUTH_MIDDLEWARE`). OAuth2 Bearer becomes the 4th method, checked via `Authorization: Bearer ` — disambiguated from HMAC by the `Bearer` scheme prefix.

**Webserver changes:**

- Add proxy routes: `/oauth2/*` → Account Manager's service address
- Same pattern as existing `/func/*` → Manager proxy and existing anonymous SSO routes (`/func/openid/*`, `/func/saml/*`, `/func/custom-auth/*`)
- Anonymous (no auth required) for authorize, token, jwks, discovery
- Authenticated routes for userinfo, revoke, introspect pass through session context
- Phase 0's `/server/authorize` remains active — apps stay on Phase 0 until Phase 2 is production-ready

**Client registration config:**

```toml
# account-manager.toml

[oauth2]
access_token_ttl = "15m"
refresh_token_ttl = "30d"
authorization_code_ttl = "60s"

[oauth2.clients.webui]
client_secret = "env:OAUTH2_WEBUI_SECRET"
redirect_uris = ["https://webserver.example.com/oauth2/callback"]
allowed_scopes = ["openid", "profile", "email", "compute:*", "admin:*"]
grant_types = ["authorization_code", "refresh_token"]
app_type = "first_party"
auto_approve = true

[oauth2.clients.my-app]
client_secret = "env:OAUTH2_MYAPP_SECRET"
redirect_uris = ["https://myapp.example.com/api/auth/callback"]
allowed_scopes = ["openid", "profile", "email", "pipelines:*"]
grant_types = ["authorization_code", "refresh_token"]
app_type = "first_party"
auto_approve = true
access_token_ttl = "30m"
```

**Scope registry:**

```
# OIDC standard
openid Authenticate and get user identity
profile User's full name and role
email User's email address

# Compute
sessions:read View compute sessions
sessions:write Create and modify compute sessions
sessions:delete Terminate compute sessions
compute:* Full compute access

# Storage (maps to RBAC VFolderPermission)
vfolders:read Read vfolder attributes and contents
vfolders:write Create vfolders, write contents
vfolders:delete Delete vfolders and contents
vfolders:mount Mount vfolders to sessions (ro/rw/wd)
storage:* Full storage access

# Images
images:read View available images
images:write Register and modify images
images:* Full image access

# Model Serving (maps to RBAC ModelDeployment/Endpoint)
model-services:read View model service deployments
model-services:write Create and manage model services
model-services:* Full model serving access

# Pipelines
pipelines:read View pipelines
pipelines:write Create and modify pipelines
pipelines:execute Run pipelines
pipeline_jobs:read View pipeline job results
pipeline_jobs:write Create and cancel pipeline jobs

# Admin (maps to RBAC Domain/Project/User permissions)
users:read View user accounts
users:write Manage user accounts
groups:read View groups/projects
groups:write Manage groups/projects
domains:read View domains
domains:write Manage domains
keypairs:read View keypairs
keypairs:write Manage keypairs
resource-groups:read View resource groups (scaling groups)
resource-groups:write Manage resource groups
admin:* Full admin access
```

> **Note:** Scope names intentionally use a flat `entity:operation` pattern rather than mirroring the full RBAC `EntityType`/`OperationType` enums (which have 60+ entity types × 9 operations). Coarse OAuth scopes are mapped to fine-grained RBAC permissions in Manager's middleware via a scope→permission mapping table.

**Acceptance criteria:**
- A client can discover endpoints via `/.well-known/openid-configuration`
- A client can complete Authorization Code + PKCE flow (via programmatic test or minimal test UI)
- JWT access token is verifiable using JWKS endpoint
- Manager API accepts `Authorization: Bearer ` and enforces scopes
- Refresh token rotation works; reuse of old refresh token revokes the grant family
- Legacy HMAC auth continues to work unchanged
- First-party clients are auto-seeded from config on Account Manager startup
- OpenID Connect conformance test suite passes for core flows
- Phase 0 apps continue working without changes

---

### Open: Account Manager ↔ Manager User Identity Sync

> **Must resolve before Phase 2.**

Account Manager and Manager maintain **separate user databases**. Phase 2 authenticates users against Account Manager's `user_profiles` table and issues JWTs with Account Manager's user UUIDs. Manager needs to resolve these identities for domain/group enforcement.

**Options:**
1. **Shared database** — Account Manager reads/writes Manager's existing `users`/`keypairs` tables directly. Simplest, but tight coupling.
2. **Sync via internal API** — Account Manager calls Manager's internal API (or vice versa) to sync user records on login. Loose coupling, eventual consistency.
3. **Manager trusts JWT claims only** — Manager extracts `domain`, `role`, `email`, `groups` from JWT claims and constructs a request context without looking up its own users table. Cleanest separation, but requires rethinking Manager's auth context model.

The chosen approach affects Phase 1 (Account Manager needs user data to issue tokens) and Phase 2 (Manager needs to resolve JWT identities).

**Investigation note:** Account Manager currently has its own `users` table (UUID only), `user_profiles` table (username, email, password, role, status), and `keypairs` table (access_key, secret_key, expired_at) — all in a separate database from Manager's `users`/`keypairs` tables. Manager's auth middleware (`_populate_auth_result`) resolves `access_key` → keypair row → user row to build `request["user"]` context including `domain_name`, `resource_policy`, etc. For Option 3, the JWT would need to carry enough claims to construct this context without a DB lookup, or Manager would need to look up its own user record by UUID/email from the JWT `sub`/`email` claim.

---

### Phase 2: Login UI + Cross-App Session

This is the phase where **apps migrate from Phase 0 to the full OIDC flow**. Phase 2 adds sessions and a production login UI, so SSO and interactive login work end-to-end.

**Account Manager:**
- Production login page rendered at `/oauth2/authorize` when no session exists
- Supports email/password authentication (validates against `user_profiles.password`)
- Session stored in Redis/Valkey (Account Manager gets its own session infrastructure)
- Session cookie: `HttpOnly; Secure; SameSite=Lax` (`Secure` configurable for HTTP-only on-prem)
- Auto-approve for first-party clients (no consent screen)
- If session exists → issue auth code immediately → redirect back (enables cross-app SSO)

**Webserver:**
- Phase 0's `/server/authorize` deprecated — apps now use `/oauth2/authorize` (proxied to Account Manager)

**Acceptance criteria:**
- User can log in via Account Manager's login page and complete OIDC flow to a first-party app
- User logged into one app visits another app → redirected to Account Manager → auto-approved instantly → SSO works
- Session timeout configurable
- Migration guide published for apps moving from Phase 0 (ID Token) to Phase 2 (Authorization Code + PKCE)

### Phase 2.5: First-Party Client Migration

> Depends on Phase 2 (production login UI + session). Can proceed incrementally per client.

This phase migrates existing first-party clients from their current custom auth patterns to standard OAuth2/OIDC.

#### FastTrack Migration

**Current state** (from investigation):
- Django app with custom `UserToken` model (Bearer tokens, 40-char hex, max 10 per user)
- `User` model stores `access_key`/`secret_key` from Manager (synced on login via `_sync_user_from_manager()`)
- Two auth classes chained: `BearerAuthentication` → `BackendAIAuthentication` (`workflow/middlewares/authentication.py`)
- Three login endpoints: `POST /auth/login/` (password), `POST /auth/keypair/` (keypair), `POST /auth/sso/token/` (sToken relay)
- Proxy layer converts Bearer→HMAC on every `/func/*` request to Manager

**Target state:**
- Register FastTrack as an OAuth2 client (confidential, `authorization_code` + `refresh_token` grants)
- Remove `UserToken` model, `User.access_key`, `User.secret_key` fields
- Remove `PasswordLoginAPIView`, `KeypairLoginAPIView`, `SSOTokenLoginAPIView` — login handled by Account Manager
- Replace `BearerAuthentication` with standard JWT validation (verify against Account Manager JWKS)
- Remove Bearer→HMAC proxy conversion — use `Authorization: Bearer ` directly to Manager
- Remove local brute-force protection (handled by Account Manager)
- Remove user sync logic (`_sync_user_from_manager`) — user identity comes from JWT claims

**What this eliminates:**
- Duplicate user database and sync drift
- Bearer→HMAC conversion overhead
- Epoch date HMAC workaround (#5737)
- Session conflicts (409) — OAuth tokens are per-client, not per-user-globally
- Dual brute-force protection
- 2FA format heuristics — Account Manager handles all 2FA

#### WebUI Migration

**Current state** (from investigation):
- Web server manages browser sessions in Redis/Valkey with access_key/secret_key stored in session
- Login via `POST /server/login` → proxied to Manager → session created
- Two connection modes: SESSION (proxy) and API (direct HMAC)
- SSO via `token_login_handler` accepting `sToken` cookie
- Session extension via `POST /server/extend-login-session`

**Target state:**
- Register WebUI as OAuth2 client (public, `authorization_code` + PKCE)
- Web server becomes BFF (Backend-For-Frontend): holds refresh token server-side, issues short-lived access tokens to browser
- Or: Web server becomes thin static file server; browser handles OIDC flow directly (SPA pattern)
- Remove `POST /server/login`, `POST /server/logout`, `POST /server/extend-login-session` — handled by OAuth2 token lifecycle
- Frontend uses access token for API requests instead of session-proxied HMAC

**Acceptance criteria:**
- FastTrack and WebUI authenticate via standard OIDC Authorization Code + PKCE
- No custom auth code remains in either client (login forms, token storage, HMAC signing)
- Cross-app SSO works between FastTrack and WebUI (user logs into one, auto-approved in other)
- Migration guide for other first-party and third-party clients

### Phase 3: Enterprise Identity Federation

- **LDAP/Active Directory**: Account Manager authenticates against corporate directory
- Bind-and-search or direct bind patterns
- Attribute mapping (sAMAccountName → email, memberOf → groups/roles)
- Periodic sync for group membership changes
- Works in air-gapped environments
- **External OIDC IdPs**: Google, Keycloak, Okta, Azure AD
- Database-backed IdP registry in Account Manager's DB
- Account Manager acts as OIDC RP to external IdPs
- Login page shows "Login with ..." buttons for configured IdPs
- User identity mapping (by email or subject) with optional auto-provisioning
- **SAML 2.0**: For enterprise IdPs that only speak SAML
- Existing hook plugins in Manager continue working during migration

**Acceptance criteria:**
- User can log in via LDAP credentials on air-gapped deployment
- User can log in via external Google/Keycloak OIDC on cloud deployment
- External IdPs configurable via database (Admin API), not code changes

### Phase 4: Single Logout

- `POST /oauth2/logout` endpoint with `id_token_hint` and `post_logout_redirect_uri`
- Account Manager session invalidation
- Optional back-channel logout notifications to registered clients

**Acceptance criteria:**
- User logs out of one app → all apps are logged out
- Back-channel logout notification delivered to registered clients

### Phase 5: Advanced Grant Types

- **Device Authorization Grant** — for CLI tools and headless environments
- `POST /oauth2/device/authorize` → returns device_code, user_code, verification_uri
- Verification page on Account Manager
- CLI polls `/oauth2/token` with device_code grant type
- **Client Credentials Grant** — for service-to-service auth

**Note:** CLI in air-gapped headless environments can continue using the existing `/auth/authorize` endpoint, which remains supported indefinitely.

**Acceptance criteria:**
- CLI tool can authenticate via Device Authorization Grant
- Service account can obtain access token via Client Credentials Grant

## Backward Compatibility

- All existing Manager `/auth/*` endpoints remain functional indefinitely
- HMAC and legacy JWT authentication continue to work in Manager API middleware
- OAuth2 Bearer is an additional auth method, checked alongside existing methods
- HMAC-authenticated requests get implicit full scope (unchanged behavior)
- Account Manager's existing `applications` table evolves into `oauth2_clients` (migration, not replacement)
- Phase 0's `/server/authorize` remains as a lightweight fallback until Phase 1 is production-ready

## Security Requirements

- PKCE required for all authorization_code grants
- RS256 (asymmetric) token signing — clients validate with public key from JWKS
- Refresh token rotation on every use; reuse detection revokes entire grant family
- Authorization code single-use
- Session cookie: `HttpOnly; Secure; SameSite=Lax` (`Secure` configurable for on-prem)
- JWKS key rotation support (multiple keys with `kid`)
- Rate limiting on token endpoint
- On-prem TLS flexibility: Allow `http://` redirect URIs when explicitly configured
- Run [OpenID Connect conformance test suite](https://openid.net/certification/testing/)

## Key Files

### Account Manager (primary implementation target — Phase 1+)

| File | Current State | Change |
|---|---|---|
| `src/ai/backend/account_manager/api/auth.py` | Stubbed (empty) | Implement all OAuth2 endpoints |
| `src/ai/backend/account_manager/api/application.py` | Stubbed (empty) | Implement client management API |
| `src/ai/backend/account_manager/models/application.py` | `ApplicationRow` with name, redirect_to, token_secret | Extend to full OAuth2 client model |
| `src/ai/backend/account_manager/models/keypair.py` | `KeypairRow` with ak/sk | Unchanged (keypair auth remains) |
| `src/ai/backend/account_manager/models/user.py` | `UserRow` (uuid only) | Unchanged |
| `src/ai/backend/account_manager/models/userprofile.py` | `UserProfileRow` with password, role, status | Unchanged (used for password auth) |
| `src/ai/backend/account_manager/server.py` | aiohttp app with /auth, /application sub-apps | Add session middleware, RS256 key loading |
| `src/ai/backend/account_manager/config.py` | `AccountManagerConfig` | Add OAuth2 config section |

### Webserver (Phase 0 + proxy for Phase 1+)

| File | Change |
|---|---|
| `src/ai/backend/web/server.py` | Phase 0: Add `/server/authorize` endpoint. Phase 1: Add `/oauth2/*` proxy routes. Existing anonymous SSO routes (`/func/openid/*`, `/func/saml/*`, `/func/custom-auth/*`) remain. |
| `src/ai/backend/web/proxy.py` | Phase 1: Add anonymous proxy handler for OAuth2 endpoints (same pattern as `anon_web_plugin_handler`) |
| `src/ai/backend/web/auth.py` | Phase 0: Add ID Token signing helper. Currently has `get_api_session()`, `get_anonymous_session()`, `generate_jwt_token_for_session()` — extend with ID Token generation. |

### Manager (JWT validation — Phase 1)

| File | Change |
|---|---|
| `src/ai/backend/manager/api/rest/middleware/auth.py` | Add OAuth2 Bearer token validation via JWKS as 4th auth method (after existing JWT/HMAC/Hook). Populate same `request["user"]`/`request["keypair"]`/`request["is_admin"]` context. |

### Common (shared types — Phase 1)

| File | Change |
|---|---|
| `src/ai/backend/common/jwt/config.py` | Extend `JWTConfig` to support RS256 algorithm alongside existing HS256 |
| `src/ai/backend/common/jwt/signer.py` | Add RS256 signing support to `JWTSigner` (currently HS256 only) |
| `src/ai/backend/common/jwt/validator.py` | Add RS256 validation + JWKS fetching/caching to `JWTValidator` |
| `src/ai/backend/common/dto/manager/auth/types.py` | Add `OAUTH2_BEARER` to `AuthTokenType` enum |

### FastTrack (Phase 2.5 — client migration)

| File | Current State | Change |
|---|---|---|
| `workflow/users/models.py` | `User` (access_key, secret_key), `UserToken` (bearer) | Remove access_key/secret_key from User, remove UserToken model |
| `workflow/users/api/views.py` | 3 login views + user sync | Remove all login views and sync logic |
| `workflow/middlewares/authentication.py` | `BearerAuthentication`, `BackendAIAuthentication` | Replace with JWT validation via JWKS |
| `workflow/proxy/views.py` | Bearer→HMAC conversion | Remove conversion; forward Bearer JWT directly |
| `workflow/users/api/bruteforce.py` | Per-username brute-force protection | Remove (handled by Account Manager) |

## References

- [OpenID Connect Core 1.0](https://openid.net/specs/openid-connect-core-1_0.html)
- [OAuth 2.0 Authorization Framework (RFC 6749)](https://www.rfc-editor.org/rfc/rfc6749)
- [PKCE (RFC 7636)](https://www.rfc-editor.org/rfc/rfc7636)
- [OAuth 2.0 Token Revocation (RFC 7009)](https://www.rfc-editor.org/rfc/rfc7009)
- [OAuth 2.0 Device Authorization Grant (RFC 8628)](https://www.rfc-editor.org/rfc/rfc8628)
- [OAuth 2.0 Token Introspection (RFC 7662)](https://www.rfc-editor.org/rfc/rfc7662)
- [JSON Web Key Set (RFC 7517)](https://www.rfc-editor.org/rfc/rfc7517)
- [OpenID Connect Conformance Testing](https://openid.net/certification/testing/)

Start by reading src/ai/backend/account_manager/ and its stubbed /auth and /application apps, then review common/jwt/, common/data/permission/types.py, and common/contexts/user.py. Done means the planned OAuth2/OIDC endpoints, RS256/JWKS and token flows, RBAC scope mapping, and webserver /oauth2/* proxy are implemented.

Written by the indexing model from the issue text.