Build the static plugin runtime and z.ai conversational adapter

Build the static plugin runtime and z.ai conversational adapter

Status: accepted source and default-off sandbox deployment are complete; the remaining secret-backed z.ai proposal/rollback evidence is consolidated into #128 after #166/#182 restore the stable canonical Card/Task baseline
Tags: enhancement, assistant, backend, infra, testing, human, P1
Depends on: #123 persistence/runtime foundation is satisfied; normal dark deployment is proven; live provider acceptance depends on the #128 HUMAN matrix after #166 A/B/C → #182 preflight → D → cleanup and one successful ordinary all-off deployment
Blocks: no source implementation — downstream slices are shipped; #125 remains a live-acceptance dependency for closing #128/#122
Next owner: #166 orchestrator/credentialed operator for storage sequencing, then the authorized #128 HUMAN z.ai/Telegram owner with On-Call; PM closes #125 from the exact cross-posted subset below
Resume condition: #166/#182 complete the reviewed live sandbox sequence, canonical writers reopen, temporary phase controls are removed, and normal all-off OIDC deployment succeeds. Then #128 privately confirms the exact secret reference and runs the bounded text/todo model stage; do not run a separate #125 provider smoke.
Parent: #122
Architecture: _docs/CONVERSATIONAL_AGENT_PLUGIN_ARCHITECTURE.md at 4e5402c

Current lifecycle evidence

Commit 543d63c51795b5a8849fc4ba0d8b7802639c51a3 is integrated with its Assistant Engineer, Tester, PM, and original On-Call acceptance. Its static registry, deterministic build/schema identity, two-turn protocol, context receipts, bounded glm-5.2 client, exact secret-reference IAM, no-retry/error redaction, and disabled-default behavior remain the accepted contract.

The original On-Call deployment passed checks, OIDC deploy, seed, and smoke with the runtime disabled, no configured z.ai secret reference, empty deployed registry, and no z.ai call. The integrated #128 all-off graph was later deployed successfully in run 31545689094, and run 31711997388 reconfirmed the stable all-off normal OIDC deploy, seed, and smoke path.

Dark deployment proves fail-closed configuration and absence of provider calls, not secret-backed skill_load/skill_invoke behavior. No HUMAN secret readback, z.ai/Telegram call, flag enablement, live proposal, provider-log inspection, or rollback has run. #128 is the sole live-evidence owner because its text/todo stage uses this exact model/runtime path and its reverse rollback proves disabled behavior.

#166 is the active rollout sequencing boundary. Although #125's model proposal is non-executing, the consolidated #128 text/todo stage proceeds to a Task effect; splitting the provider smoke ahead of the approved Tasks replacement would create partial rollout evidence on the wrong storage baseline.

Scope

Build the channel-independent model/runtime boundary used by later Telegram and
domain slices:

1. a trusted, build-time TypeScript plugin registry;
2. a compact plugin catalog and progressive disclosure;
3. the exact two-model-turn skill_load → skill_invoke protocol;
4. deterministic plugin/build/schema identity and stale-turn rejection;
5. bounded context assembly with a persisted context receipt;
6. a reusable z.ai client for the Anthropic-compatible Messages API;
7. disabled-by-default deployed configuration and least-privilege z.ai secret
   wiring.

This child proves the framework with injected fake plugins and executors. The
deployed registry may remain empty until the todo child registers the first
domain plugin. Nothing in this issue writes a task, Typefully draft, document,
or other domain resource.

The existing socialDraftAssistant.ts z.ai code is evidence for the provider
contract, not the reusable runtime boundary. The shared adapter must not depend
on social-post types or call Typefully. Later work may migrate that caller; this
issue must not change the current social workflow's product behavior.

Product decisions

These choices keep the MVP small and deterministic:

• Registry discovery is a static TypeScript list. There is no package scanning,
  runtime installation, marketplace, semantic catalog search, or flow DSL.
• At most one mutation plugin is active. Multi-plugin mutation planning is
  rejected.
• The core owns conversation/session actions, permissions, validation
  sequencing, proposal boundaries, and model tool definitions. A plugin cannot
  add approval/cancellation/session transitions or grant itself authority.
• The initial core permission allowlist is exactly
  todo:create:self and typefully:create-saved-draft. They are contracts for
  later children, not executable capabilities in this issue.
• admin and operator remain the only recognized roles. Registry
  configuration may narrow a plugin by role and channel, but cannot widen a
  core permission. Disabled, wrong-role, and wrong-channel plugins are omitted
  from the catalog and cannot be loaded by name.
• No automatic provider retry is performed. A new operator event may safely
  start a new turn after a transient failure.
• A model turn may make at most one framework tool call. Invalid or malformed
  output does not trigger an unbounded correction loop.

Plugin contract and registry

Define a typed plugin contract containing:

id
version
displayName
summary
activationHints
skillInstructions
actions[]
  name
  description
  strict input schema
  effect: read | proposal
  corePermission
  reconciliationMode (required for a future external proposal effect)
validator
proposal renderer (required for a proposal effect)
executor/reconciler declarations (validated as metadata here; not invoked here)
buildDigest
schemaDigest

Registry construction/startup must fail with a safe configuration error for:

• duplicate or non-canonical plugin/action IDs;
• a missing required field or handler declaration;
• a schema that permits unknown properties;
• duplicate action names;
• an unknown core permission;
• a read action that declares a mutation executor;
• a proposal action without a renderer or permission;
• an external proposal action without one of
  provider_idempotency, correlation_lookup, or
  operator_reconciliation_only;
• an invalid build/schema digest;
• an enabled role or channel outside the core allowlists.

The compact catalog exposes only id, displayName, one short summary, and
bounded activation hints for plugins available to the current actor/channel.
It never exposes full instructions, schemas, handlers, permissions, secrets, or
runtime configuration.

Build and schema digests

• Both digests use lowercase sha256:<64 hex characters>.
• schemaDigest is calculated at build/startup from canonical JSON containing
  the ordered action names, effects, core permissions, reconciliation modes,
  and strict input schemas.
• buildDigest is generated, not hand-maintained. A deterministic build helper
  hashes the compiled plugin module plus its canonical manifest/contract and
  emits registry metadata. Re-running it over identical input must produce the
  same digest; changing instructions, schema, renderer, validator, executor, or
  reconciler code must change it.
• Registry startup recomputes/verifies generated metadata and fails closed on a
  mismatch.
• A loaded skill and every later proposal record the exact build and schema
  digests. If the build disappears or changes, unclaimed work is stale and must
  be regenerated; this issue does not build the executor-version retention
  mechanism owned by the approval/execution child.

Exact two-turn protocol

The following is the only plugin-selection protocol in the MVP.

Turn 1: select and load

The runtime assembles:

• core system/approval rules;
• the actor/channel-filtered compact catalog;
• the bounded summary and recent conversation events supplied by #123;
• only the skill_load framework tool:

{
  "plugin": "<canonical plugin id>"
}

If the model returns ordinary assistant text because it cannot yet select a
plugin, the runtime returns that clarification and makes no second provider
call. If it calls skill_load, the runtime requires exactly one tool call,
validates catalog availability, role, channel, and the current conversation
revision, then stores a SkillLoadReceipt containing:

conversationId
conversationRevision
pluginId
pluginBuildDigest
schemaDigest
opaque loadNonceHash
createdAt
expiresAt
status: active | consumed | invalidated | expired

The opaque nonce expires after five minutes and is invalidated by any newly
committed conversation event or revision. The successful skill_load ends
turn 1; its provider response cannot also invoke the plugin.

Turn 2: invoke the loaded skill

The runtime starts one separate z.ai Messages call containing:

• the same higher-priority core rules;
• the selected plugin's instructions and action schemas;
• the bounded conversation context;
• the exact plugin/build/schema identity and opaque load nonce;
• only the skill_invoke tool:

{
  "plugin": "<same plugin id>",
  "action": "<registered action>",
  "input": {},
  "load_nonce": "<opaque nonce>"
}

The runtime accepts exactly one skill_invoke call. Before plugin validation
it verifies the authenticated actor/channel policy again, conversation ID and
revision, plugin ID, build digest, schema digest, nonce status/expiry, and action
effect/permission. The input is then validated against the strict registered
schema before any plugin handler is reached.

A valid invocation atomically consumes the load receipt through #123's
conditional persistence primitive. It returns only a typed, non-executing
result such as missing-field clarification, validated draft state, or a pure
proposal candidate. M2 tests use fakes for all three. It never claims approval
or invokes an executor.

An invalid schema/tool/action/nonce or a provider response containing text plus
a tool call is rejected. There is no automatic third provider call. The user
gets a safe retry/clarification interaction; a later input starts with a fresh
revision/load receipt. Duplicate delivery returns the stored consumed result
when available and never invokes a fake handler twice.

Bounded context

Implement a deterministic ContextAssembler with an injectable token counter.
The deployed runtime uses a documented conservative estimator
ceil(UTF-8 byte length / 3) because z.ai does not expose a compatible
pre-flight tokenizer. Tests use an exact fake counter.

Defaults:

• maximum estimated input: 16,384 tokens;
• maximum provider output: 4,096 tokens;
• summary: at most 2,048 input tokens;
• recent events: at most 8,192 input tokens, selected newest-first but emitted
  in chronological order and never split mid-event;
• source excerpts: at most 4,096 input tokens (the M2 runtime supplies none);
• proposal-state reference: at most 512 input tokens.

Core rules, the filtered compact catalog, and—during turn 2—the one active
plugin contract are mandatory. If mandatory content alone exceeds the total,
fail closed as configuration error instead of silently truncating policy or
schemas. Optional layers are dropped/truncated in this order: oldest recent
events, source excerpts, then summary. Complete history, raw proposal payloads,
approval records, authentication records, secrets, and unrelated plugin
schemas never enter context.

For every provider call, persist through #123 a small ContextReceipt with:

algorithmVersion
conversationId and source revision
summary checkpoint ID/hash, if used
included event sequence ranges/hashes
source reference/revision hashes, if used
plugin ID/build/schema digests, if loaded
estimated input count and configured limits
truncation flags
provider/model identifier

The receipt contains no prompt/completion bodies or secret values.

z.ai Anthropic-compatible client

Provide an injected conversational model interface and one deployed z.ai
implementation.

Defaults:

• base URL: https://api.z.ai/api/anthropic;
• request URL: https://api.z.ai/api/anthropic/v1/messages;
• model: glm-5.2;
• header: anthropic-version: 2023-06-01;
• authentication header: x-api-key;
• request timeout: 20 seconds;
• maximum output: 4,096 tokens;
• at most two provider calls per incoming operator event.

Configuration accepts the base URL, model, timeout, and limits, but deployed
startup rejects non-HTTPS URLs and any host other than api.z.ai. Tests may
inject a local fake transport without weakening deployed validation.

The client must:

• serialize Anthropic-compatible system, messages, tools, and
  tool_choice payloads;
• accept only expected tool_use blocks and expected stop reasons;
• reject multiple tools, unknown tools/actions, unexpected mixed text/tool
  output, malformed JSON, oversized provider payloads, and limit violations;
• use AbortController for the deadline;
• expose typed safe error codes such as model_timeout,
  model_rate_limited, model_unavailable, model_invalid_output,
  model_policy_rejected, and model_config_error;
• never return raw provider error bodies to a user.

There is no retry inside the client. Timeout, HTTP 429/5xx, malformed output,
and network ambiguity create no draft/proposal and return a recoverable,
operator-readable message. Provider output remains untrusted and cannot
authorize, approve, or execute an action.

Secrets, logging, and deployed wiring

Add a dedicated out-of-band Secrets Manager reference for the conversational
z.ai key. The managed secret value is JSON:

{ "apiKey": "<z.ai key>" }

Use a SAM parameter such as ZaiConversationalApiKeySecretArn and expose only
that reference to the backend (for example,
ZAI_CONVERSATIONAL_API_KEY_SECRET_ARN). Grant
secretsmanager:GetSecretValue only for that exact ARN. Resolve/cache the key
inside the deployed client; do not copy it into a CloudFormation plaintext
parameter, GitHub secret value, ordinary Lambda environment value, model
context, or generated registry metadata.

The conversational runtime remains disabled by default until rollout. Empty
secret configuration is allowed only while disabled. Enabling with no valid
secret ARN must fail closed. The existing OIDC deploy workflow may pass the
non-secret ARN as deployment configuration, but must never retrieve or echo the
key.

Redaction rules:

• do not log/store prompt bodies, completion bodies, plugin input, conversation
  text, provider bodies, API keys, authorization headers, or secret ARNs;
• logs/metrics may contain correlation ID, safe error code, duration, HTTP
  status class, model, estimated/input/output token counts, plugin ID, digests,
  and hashes;
• recursively redact keys matching secret/token/password/credential/cookie/
  authorization/API-key patterns and values matching bearer/key/signature
  patterns before any error crosses the client boundary;
• cap all safe summaries and reject overlarge model/provider fields.

Automated tests use fake transports and fake Secrets Manager only. They must not
read a developer shell key or make a network request.

Acceptance criteria

• A typed static registry and compact actor/channel-filtered catalog exist;
  startup rejects every invalid registry case listed above.
• The only initial core permissions are todo:create:self and
  typefully:create-saved-draft; plugins cannot define permissions or core
  transitions.
• Plugin build and schema digests are deterministic, verified, and change
  when their covered inputs change.
• skill_load and skill_invoke follow the exact separate-turn protocol;
  load expiry, revision, nonce, plugin, action, role, channel, build, and
  schema bindings are enforced before a fake handler runs.
• One model response cannot both load and invoke; a turn has at most one
  framework tool call and an input event makes at most two provider calls.
• A valid invocation produces only a typed non-executing result. No runtime
  path in this issue invokes an executor or mutates a domain resource.
• Duplicate invocation delivery is idempotent; stale, expired, consumed,
  mismatched, malformed, or unauthorized invocation executes no fake
  handler.
• Context assembly enforces the declared budgets and ordering, excludes
  forbidden state/unrelated plugins, and stores a body-free context receipt.
• The reusable z.ai client uses the declared Anthropic-compatible endpoint,
  model/header/tool contracts, timeout, output limit, and injected transport.
• Provider/network/output/config failures use bounded safe codes, create no
  proposal, and leak no prompt, completion, plugin input, raw provider body,
  key, token, secret reference, or authorization header.
• SAM adds disabled-by-default z.ai configuration, an exact Secrets Manager
  reference, and least-privilege read permission without embedding the key.
• Existing unrelated APIs and the current social-draft behavior remain
  compatible while the new runtime is disabled.
• Automated tests make zero live z.ai calls and require no real secret.
• Normal OIDC deployment proves the runtime remains disabled by default,
  needs no key while disabled, performs no z.ai call, and preserves existing
  routes/behavior.
• #166 A/B/C, #182's no-replacement Cards preflight, D, and cleanup complete;
  canonical writers reopen, temporary phase controls are deleted, and one
  following normal all-off deployment succeeds before the consolidated live
  #128 model/todo stage.
• [HUMAN via #128] An authorized operator privately confirms the sandbox
  z.ai secret exists in the accepted JSON shape and configures only its exact
  ARN through normal OIDC/SAM; no key, ARN, full environment map, or secret
  metadata is posted publicly.
• [HUMAN via #128] One bounded private glm-5.2 text/todo journey performs
  exactly separate skill_load then skill_invoke model turns, at most two
  provider calls for the operator event, and yields one typed non-executing
  proposal before any approval/domain effect. No unrelated plugin/schema or
  provider retry is observed.
• [HUMAN via #128] Sanitized observation shows only allowlisted model, safe
  status/error category, duration/count, plugin ID, and digest/hash metadata;
  no prompt, completion, plugin input, conversation text, API key, secret
  reference, auth header, raw provider body, or private identifier leaks.
• [HUMAN via #128] Reverse OIDC rollback disables the runtime/ingress and
  reconfirms readiness/fail-closed behavior with zero new secret lookup or
  provider call; no manual Lambda edit or key rotation/deletion is required.
• #128 cross-posts the exact sanitized evidence links above. PM verifies no
  separate #125 smoke and no missing criterion; #125 may then close even if
  #128 todo execution, media, Typefully, alarm, or other rollout stages
  remain open.

Test scenarios

Registry validation and catalog filtering

Given valid fake plugins plus duplicate IDs/actions, permissive schemas, unknown
permissions, missing handlers, invalid digests/reconciliation modes, disabled
plugins, and role/channel restrictions
When the registry starts and catalogs are requested
Then invalid registries fail safely and each actor sees only the bounded public
metadata for permitted plugins.

Deterministic digests

Given identical and individually changed fake plugin source/manifest/schema
fixtures
When generated metadata is built and verified twice
Then identical input has identical digests, each covered semantic/code change
changes the correct digest, and stale/tampered metadata fails startup.

Separate model turns

Given a fake z.ai transport that records requests
When turn 1 calls skill_load
Then it receives only the catalog/load tool, the response ends the call, and
turn 2 is a new request containing only the selected plugin contract and
skill_invoke; unrelated schemas are absent.

Stale and replayed loads

Given concurrent input that advances the conversation revision, a five-minute
expiry, a consumed nonce, a duplicate delivery, a changed plugin build/schema,
and wrong actor/channel/action values
When skill_invoke is validated
Then stale or mismatched calls produce no handler invocation, while exact
duplicate delivery returns the recorded result and the valid path runs once.

Invalid model output

Given unknown/multiple/mixed tool calls, extra schema fields, malformed blocks,
oversized bodies, wrong stop reasons, and missing required input
When either model response is parsed
Then no automatic third call occurs, no proposal or executor is reached, and a
safe recoverable interaction is returned.

Context budgeting

Given oversized summaries, old/recent events, source excerpts, proposal state,
unrelated plugin schemas, secret-like text, and an exact fake token counter
When each turn is assembled
Then mandatory policy/schema remains intact, optional content is reduced in the
specified order, whole recent events remain chronological, the hard budget is
not exceeded, forbidden content is absent, and the context receipt exactly
describes the included revisions/hashes and truncation.

Provider contract and failures

Given fake success, timeout, cancellation, network failure, 401/403, 429, 5xx,
invalid JSON, malicious error bodies, and secret-like nested values
When the z.ai client runs
Then URL/headers/body/tool choice are exact, there is no retry, errors map to
safe codes, and captured logs/results contain none of the sensitive values or
prompt/completion bodies.

Secret and infrastructure boundary

Given the SAM template, disabled/enabled runtime configurations, a fake secret,
and missing/malformed secret values
When infrastructure validation and client startup run
Then disabled deployment needs no key, enabled runtime fails closed without
one, only the backend role can read the exact ARN, and no plaintext key appears
in templates, environment, build output, tests, or workflow logs.

Backward compatibility

Given the new runtime is disabled
When the existing backend assistant/Telegram/API tests run
Then existing behavior and routes are unchanged and no new z.ai secret lookup
or model call occurs.

Required verification

Run:

npm --prefix backend test
npm --prefix backend run typecheck
npm --prefix backend run build
make sam-validate
make sam-build

Tester must additionally inspect generated SAM for the exact secret ARN
permission/environment reference and run focused race/replay, context-budget,
digest, provider-contract, and recursive-redaction tests. No Playwright run or
screenshots are required because this issue changes no portal UI.

Assistant Engineer reviews the runtime/provider boundary before Tester
acceptance. After shipping, On-Call monitors the existing OIDC deployment. The
real secret/provider call remains the [HUMAN] gate; if all agent-verifiable work is
complete first, commit with Refs #125, add human, and leave the issue open
per process.

Dependencies and closure ownership

#123's conversation revisions, append-only events, conditional persistence, summary checkpoints, and receipt storage are satisfied source prerequisites, not current blockers. The sandbox z.ai key remains an out-of-band HUMAN configuration; a historical development-key smoke is not deployed evidence and must not be repeated or copied here.

#128 owns the only remaining live provider acceptance. #125's closure subset is the exact secret-reference configuration, one successful two-turn proposal path, safe metadata/redaction, and disable rollback above. It does not require todo approval/execution, voice/photo, Typefully external execution, alarm canaries, or every #128 stage. Those remain their owning child/rollout criteria.

#166/#182 are sequencing dependencies for the consolidated rollout, not because #125 writes Tasks. They do not change the plugin/runtime contract and do not justify compatibility or migration machinery in #125.

The public repository contains runtime code, schemas, infrastructure, and sanitized fixtures only. Operational prompts, provider configuration, secret metadata, private conversations, and knowledge remain outside public source/evidence.

Out of scope

• Todo, Typefully, SOP, podcast, workflow, recurring-work, document, memory, or
  any other deployed domain plugin.
• Any executor invocation, approval/presentation token, durable execution
  worker, provider reconciliation, or domain mutation.
• Telegram/web channel adapters, voice, photos, uploads, group behavior, or
  cross-channel continuation.
• Dynamic plugins, package discovery, plugin marketplace, semantic catalog
  search, general flow DSL, arbitrary MCP/tool proxying, or multi-plugin
  mutation plans.
• History-wide retrieval, durable personal memory, embeddings, or source
  document ingestion.
• Migrating the existing social-draft flow to the new runtime or changing its
  current Typefully behavior.
• z.ai native multimodal/vision and Groq transcription; those belong to #121.
• Modifying ../dtc-operations, ../datatasks, or ../podcast-assistant.

Read _docs/CONVERSATIONAL_AGENT_PLUGIN_ARCHITECTURE.md at 4e5402c, then review the accepted implementation and the evidence consolidated in #128. Follow the #166/#182 sequencing and the linked all-off deployment runs; completion is the authorized #128 HUMAN z.ai/Telegram evidence and successful normal all-off deployment, not a separate provider smoke.