langgenius / langgenius/dify

[Feature Request] Agent-scoped persistent storage for runtime-generated data (workspaces are conversation-scoped, published runs are read-only)

Open
#41,375 1 comment 1 reaction 0 assignees View on GitHub
1.17.0
Dominant language
TypeScript
Stars
156k
Forks
24.6k
Avg merge
22h 9m
Merged PRs (30d)
610

Description

### Self Checks

- [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542).
- [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.
- [x] I confirm that I am using English to submit this report, otherwise it will be closed.
- [x] Please do not modify this template :) and fill in all the required fields.

### 1. Is this request related to a challenge you're experiencing? Tell me about your story.

We are evaluating the new agent stack on self-hosted Dify 1.17.0 (agent-backend + agent-local-sandbox + the `dify-agent` CLI). Our use case is a "daily report" agent: each run produces a report file (e.g. `reports/2026-08-27.md`), and a later run should aggregate the accumulated dailies into a weekly summary.

While reviewing the source to design this workflow, we found that agents have **no agent-scoped persistent storage for runtime-generated data**:

1. **Workspaces are conversation-scoped.** In `api/core/app/apps/agent_app/session_store.py`, `AgentAppSessionScope.workspace_owner` resolves to `AgentWorkspaceOwnerType.CONVERSATION` (owner_id = conversation_id) or `BUILD_DRAFT`. Files written during one conversation's run persist across that conversation's messages, but a **new conversation starts with a brand-new empty workspace** — the dailies written earlier are invisible to it.
2. **Config mutations are build-draft only.** `dify-agent config skills/files/env/note push` is rejected for published runs (`api/services/agent_config_service.py` → `_require_writable()`: *"config push is only allowed for build drafts"*). The runtime prompt documents this too (`dify-agent/src/dify_agent/layers/config/layer.py`, `_CONFIG_CLI_USAGE_PROMPT`): local edits are not saved by themselves.
3. **Home snapshots are only taken at build-apply** (`api/services/agent/home_snapshot_service.py`, `create_for_build_apply`), so anything a published run leaves in `$HOME` is not carried forward either.
4. The only generic persistence primitive available inside a published run is `dify-agent file upload` (a tool file), which has no runtime listing/enumeration surface, so an agent cannot discover its own previously uploaded artifacts.

The net effect: the build-draft → apply cycle is a great **controlled release channel for config/skills**, but there is no built-in home for **accumulated runtime data** that should survive across conversations, sessions, and re-publishes.

### 2. Additional context or comments

Source references above are from the `1.17.0` tag.

What we ended up doing (workarounds, for context): keeping everything inside one long-running conversation (fragile — see the generation-mismatch problem when re-publishing), or pushing data out to a knowledge base / external storage ourselves. Skills stay what they are today (versioned instruction packages) — we think that is correct; the gap is a proper **agent-owned data surface**.

Possible directions:

- an "agent data directory" bound read/write into every run of the agent (isolated per agent, optionally per user), and/or
- runtime append/query APIs to agent-owned storage (e.g. dataset-backed) that work in published runs without the build-draft → apply cycle.

Related but different from #39316 (global skill management): this request is about **runtime data persistence**, not skill reuse.

### 3. Can you help us with this feature?

- [ ] I am interested in contributing to this feature.

(willing to help review/design, but not able to submit a PR right now)

Contributor guide

Open the contributing guide

Research direction

Start by tracing AgentAppSessionScope.workspace_owner in api/core/app/apps/agent_app/session_store.py, then read _require_writable() in api/services/agent_config_service.py and create_for_build_apply in api/services/agent/home_snapshot_service.py. Compare these paths with the dify-agent file upload and config CLI entry points. Done should mean published runs can persist and discover agent-owned runtime data across conversations, sessions, and re-publishes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, typescript
Domain
api, backend, cli
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.