Dogfood Mu through Malten: optional agents, grounded photo reflections and moderation

Malten should be able to use Micro's managed agent at POST /agent/malten for reflection and source investigation, instead of maintaining its own Anthropic orchestration. Keep the existing HTTP endpoint and Go implementation; no new MCP model tool or separate agent service is needed.

Product goal: real-world dogfooding for Mu

Malten is a real consumer of Mu's agent platform: an anonymous, minimal capture-and-reflection application with text, photos, offline use and durable background work. This should demonstrate that another application can use Mu's standing instructions, models, tools and agent execution without rebuilding that infrastructure or adopting Mu's chat UI.

Two operating modes

• No API keys: start Malten as a personal timeline. Capture text and photos, navigate streams, and retain local memories/offline drafts. No agent loops, model calls or background source polling. Core capture must work without Mu or Anthropic. Personal content must not be published into shared public streams without an available moderation service.
• Mu configured: add a server-side Mu credential to enable the configured Malten agent and reflection capabilities. The target is one Mu integration for intelligence; a separate Anthropic key should not be a permanent requirement. Public sharing becomes available only with the required text-and-image moderation operation. An unavailable agent, exhausted quota or missing key must not prevent personal capture or discard it.

These are target behaviours, not a claim that Malten already implements the no-key mode. Its current source loops start with the application. The migration must make that distinction explicit while preserving its simple startup.

Why a hosted Malten agent

A dedicated Malten agent holds its trusted standing purpose, Islamic foundation, writing instructions and restricted tool access. It interprets the submitted thoughts/photos, searches Reminder and Aslam when relevant, incorporates current real-world information when useful, and returns a concise, referenced reflection.

Micro can serve as a general real-time information specialist; a dedicated Malten agent supplies the consistent application-specific purpose. Calling Micro for information and delegating reflection to a configured Malten agent are different responsibilities. Malten should not have to duplicate Mu's routing, tool execution or investigation loop.

Malten retains the UI, streams, capture, offline/local storage, durable jobs, moderation enforcement and publication. Mu owns the delegated intelligence. Local agent code is currently built into Malten, not separately installed plugins; once equivalent behaviour is available through Mu, redundant local orchestration can be removed. Do not recreate every local agent as a hosted agent unless needed.

Credentials, guest access and anonymity

A server credential must not require each Malten participant to create a Mu account. Anonymous participation in Malten and authenticated application-to-platform calls are compatible. Keep the key server-side and scope it to the required agent/tools.

Clarify and document the guest interface as well. The reported unauthenticated browser request is POST /agent with:

{"context": {}, "prompt": "hello", "attachment": "", "history": [], "context_id": "", "agent": "", "cards": true}

Verify which handler supports this contract, its response format, attachment support, quota identity/limits, retention, and whether third-party server-to-server use is supported. Distinguish it from the authenticated managed-agent API inspected below; do not assume the two contracts are interchangeable. Also clarify whether guest callers can select a configured agent or only the default agent.

Guest access could support an explicitly enabled trial if Mu supports that use. It must not silently enable agents in Malten's no-key mode. Do not rely on guest quota as unbounded production capacity or rotate identifiers to evade limits. Document whether requests proxied through one Malten server share a quota. The production target is a scoped application credential with predictable limits and clear failures.

Additional dogfooding acceptance

• Start Malten with no keys: personal text/photo capture and local memories work; no agent/source polling occurs and no unmoderated public publication occurs.
• Configure Mu: submit a reflection through Malten using the hosted agent's standing instructions and real tools, with no duplicate local investigation loop.
• Submit work, put the phone away or disconnect, and return to find the completed reflection in its original stream. Retry without duplicate work/publication.
• Remove the key, exhaust quota or simulate a Mu outage: capture still works, content remains safe, and unavailable AI work has a clear status.
• Demonstrate independent anonymous streams under one application credential with no cross-stream history or memory leakage.
• Complete the text-and-image moderation contract before declaring the one-key mode complete or removing the direct moderation fallback.
• Record the actual supported guest/authenticated contracts and remaining gaps. Validate through Malten itself, not only an isolated curl or MCP tool test.

Current state

At the initial inspection of agent/api.go and agent/client_context.go (recheck as Mu evolves):

• The endpoint accepts prompt / legacy text, thread, and transient location/timezone context.
• There is no image input. Prompt text is silently truncated at 8,000 bytes.
• Calls go through Ask and produce a conversation/thread.
• The response contains text, thread, agent and flow identifiers, but no structured source references.

Malten already owns capture, offline submission, durable summary jobs, streams, local timeline memories, moderation and publication. It currently uses Mu's read-only news/place/weather MCP tools with MU_API_KEY. Those live tool tests passed. The remaining move is model execution and investigation.

Required changes

1. Images and longer input. Accept text plus up to three JPEG images, associated with capture IDs, and allow a photo-only request. Malten supports 20,000 characters per post and up to 60,000 characters across 40 captures per summary. Images are at most 400 KiB each before base64 encoding. Define bounded request/model limits, preserve Unicode, and return a clear error for excess input rather than truncating it. Ensure images actually reach the chosen vision-capable model; reject unsupported models explicitly.

2. A scoped, stateless request mode. Malten supplies the relevant captures each time. Add an explicit mode that does not create or reuse conversations, write capture content into agent/account memory, or retain it in workflow/tool logs. Document any operational/provider retention that remains. Unrelated anonymous streams share a service credential and must never share personal context. Malten's new local memories are device-local; this is not permission to upload or remember them in Micro. Preserve existing conversational behaviour for ordinary clients.

3. Standing Malten instructions and tool scope. Configure the hosted Malten agent with the existing Islamic foundation and writing instructions, as trusted configuration rather than instructions embedded in untrusted captures. Foundation: Allah as Creator; purpose, worship, life's tests, accountability and return to Him; never claim to know Allah's particular reason for someone's experience. Retrieve relevant Quran/hadith and explanatory sources through Reminder/Aslam; use news/weather when relevant. Preserve source provenance and distinguish interpretation. Describe ideas and visible scenes directly, without “the user shared”, advice, conversational padding or invented feelings. Allowlist public read-only retrieval tools; exclude the service account's personal mail, files, calendar, tasks and write tools. Do not infer precise location from photos.

4. A usable response contract. Provide a compact summary plus optional sourced context and structured references (ID, title, URL, and retrieved supporting text/excerpt where needed for validation). References must come from actual tool results, not merely URLs generated by the model. Preserve source dates, uncertainty and partial-source failures. A plain summary is acceptable when no relevant supporting evidence is found. JSON/structured output must be validated before returning success.

5. Reliable server-to-server operation. Document timeouts and return machine-readable authentication, size, rate-limit, model and tool errors, including Retry-After where appropriate. Support an idempotency key scoped to account/agent/request with a documented bounded lifetime, so a lost HTTP response does not trigger duplicate charged work. This must respect the stateless retention contract. Malten already runs durable jobs independently of the browser: do not build another queue, chat UI or notification system just for this integration.

Acceptance and migration

• Exercise a captionless photo and mixed photos/text; a photo must never become “empty input” at a later stage.
• Exercise a long Unicode reflection without truncation, and reject oversized requests.
• Prove isolation between two requests/streams under the same credential, and absence of capture content from application persistence in stateless mode.
• Test source failure, fabricated reference rejection, restricted tools, unsupported image models and lost-response retries.
• Run a live text and photo request through the configured Malten agent and verify referenced output and tool activity.
• Once the contract works, switch Malten's summary worker to this HTTP endpoint and remove redundant local investigation code. Keep its offline queue, local memories, public expiry and publishing behaviour.
• Keep direct moderation during that first cutover. Removing Anthropic entirely is a separate acceptance step: provide a scoped text-and-image moderation operation with explicit allow/block/error results before switching it.

Existing implementation to carry over

• Shared foundation and investigation loop
• Reflection instructions, image handling and result validation
• Durable summary jobs
• Moderation policy
• Micro integration notes

Please implement the smallest extension of the existing agent endpoint that meets this contract, preserving existing API clients.

Start with agent/api.go and agent/client_context.go, then compare the existing behavior with agent/research.go, agent/reflection/reflection.go, and server/summary.go. Recheck the endpoint and live tool behavior before implementing the smallest compatible contract; done means stateless text/image requests, validated references, bounded errors and retries, isolation, and Malten's summary worker using the endpoint.