[Feature Request] Agent-scoped persistent storage for runtime-generated data (workspaces are conversation-scoped, published runs are read-only)
- Dominant language
- TypeScript
- Stars
- 156k
- Forks
- 24.6k
- Avg merge
- 22h 9m
- Merged PRs (30d)
- 610
Description
### Self Checks
- [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542).
- [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.
- [x] I confirm that I am using English to submit this report, otherwise it will be closed.
- [x] Please do not modify this template :) and fill in all the required fields.
### 1. Is this request related to a challenge you're experiencing? Tell me about your story.
We are evaluating the new agent stack on self-hosted Dify 1.17.0 (agent-backend + agent-local-sandbox + the `dify-agent` CLI). Our use case is a "daily report" agent: each run produces a report file (e.g. `reports/2026-08-27.md`), and a later run should aggregate the accumulated dailies into a weekly summary.
While reviewing the source to design this workflow, we found that agents have **no agent-scoped persistent storage for runtime-generated data**:
1. **Workspaces are conversation-scoped.** In `api/core/app/apps/agent_app/session_store.py`, `AgentAppSessionScope.workspace_owner` resolves to `AgentWorkspaceOwnerType.CONVERSATION` (owner_id = conversation_id) or `BUILD_DRAFT`. Files written during one conversation's run persist across that conversation's messages, but a **new conversation starts with a brand-new empty workspace** — the dailies written earlier are invisible to it.
2. **Config mutations are build-draft only.** `dify-agent config skills/files/env/note push` is rejected for published runs (`api/services/agent_config_service.py` → `_require_writable()`: *"config push is only allowed for build drafts"*). The runtime prompt documents this too (`dify-agent/src/dify_agent/layers/config/layer.py`, `_CONFIG_CLI_USAGE_PROMPT`): local edits are not saved by themselves.
3. **Home snapshots are only taken at build-apply** (`api/services/agent/home_snapshot_service.py`, `create_for_build_apply`), so anything a published run leaves in `$HOME` is not carried forward either.
4. The only generic persistence primitive available inside a published run is `dify-agent file upload` (a tool file), which has no runtime listing/enumeration surface, so an agent cannot discover its own previously uploaded artifacts.
The net effect: the build-draft → apply cycle is a great **controlled release channel for config/skills**, but there is no built-in home for **accumulated runtime data** that should survive across conversations, sessions, and re-publishes.
### 2. Additional context or comments
Source references above are from the `1.17.0` tag.
What we ended up doing (workarounds, for context): keeping everything inside one long-running conversation (fragile — see the generation-mismatch problem when re-publishing), or pushing data out to a knowledge base / external storage ourselves. Skills stay what they are today (versioned instruction packages) — we think that is correct; the gap is a proper **agent-owned data surface**.
Possible directions:
- an "agent data directory" bound read/write into every run of the agent (isolated per agent, optionally per user), and/or
- runtime append/query APIs to agent-owned storage (e.g. dataset-backed) that work in published runs without the build-draft → apply cycle.
Related but different from #39316 (global skill management): this request is about **runtime data persistence**, not skill reuse.
### 3. Can you help us with this feature?
- [ ] I am interested in contributing to this feature.
(willing to help review/design, but not able to submit a PR right now)
Contributor guide
Research direction
Start by tracing AgentAppSessionScope.workspace_owner in api/core/app/apps/agent_app/session_store.py, then read _require_writable() in api/services/agent_config_service.py and create_for_build_apply in api/services/agent/home_snapshot_service.py. Compare these paths with the dify-agent file upload and config CLI entry points. Done should mean published runs can persist and discover agent-owned runtime data across conversations, sessions, and re-publishes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, typescript
- Domain
- api, backend, cli
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100