bug: plugin upgrade fails when .difypkg file missing from disk (PVC recreation)

Self Checks

• I have read the Contributing Guide and Language Policy.
• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report, otherwise it will be closed.
• Please do not modify this template :) and fill in all the required fields.

Dify version

1.13.2 (also confirmed on latest main)

Cloud or Self Hosted

Self Hosted (Kubernetes)

Steps to reproduce

1. Install several plugins from the Marketplace (e.g. openai, anthropic, deepseek, etc.)
2. Delete or recreate the PVC backing /app/storage/plugin_packages/ on the plugin-daemon pod (simulating storage loss, node migration, or PVC reprovisioning)
3. Restart the plugin-daemon pod
4. Wait for the periodic plugin auto-upgrade check to run (process_tenant_plugin_autoupgrade_check_task), or trigger a manual upgrade from the UI

✔️ Expected Behavior

When the .difypkg file is missing from disk, the upgrade flow should detect this and re-download the package from the Marketplace, even if the DB/Redis declaration still exists.

❌ Actual Behavior

All plugins that had their package files removed show the error:

failed to move plugin to installed bucket: failed to get package file when trying to install plugin to local
open /app/storage/plugin_packages/langgenius/openai:0.3.4@3345b59...: no such file or directory

This affects every plugin whose .difypkg file was lost. In our case, 30+ plugins were affected simultaneously after a PVC recreation.

Root Cause

There are two affected code paths:

1. UI manual upgrade — PluginService.upgrade_plugin_with_marketplace() in api/services/plugin/plugin_service.py:

try:
    manager.fetch_plugin_manifest(tenant_id, new_plugin_unique_identifier)
    # already downloaded, skip
    marketplace.record_install_plugin_event(new_plugin_unique_identifier)
except Exception:
    pkg = download_plugin_pkg(new_plugin_unique_identifier)
    ...

fetch_plugin_manifest only queries the daemon DB for a PluginDeclaration record — it does not verify the .difypkg file on disk. When the file is missing but the DB record remains, the try block succeeds, the download is skipped, and the upgrade fails.

2. Auto-upgrade celery task — process_tenant_plugin_autoupgrade_check_task() in api/tasks/process_tenant_plugin_autoupgrade_check_task.py:

This task calls manager.upgrade_plugin() directly without any file existence check at all. It never verifies that the package file is present before asking the daemon to perform the upgrade.

Proposed Fix

Add decode_plugin_from_identifier() calls to verify the package file exists on disk before proceeding with upgrades:

Path 1 — plugin_service.py: Add a decode_plugin_from_identifier call after fetch_plugin_manifest:

try:
    manager.fetch_plugin_manifest(tenant_id, new_plugin_unique_identifier)
    manager.decode_plugin_from_identifier(tenant_id, new_plugin_unique_identifier)
    marketplace.record_install_plugin_event(new_plugin_unique_identifier)
except Exception:
    pkg = download_plugin_pkg(new_plugin_unique_identifier)
    ...

Path 2 — auto-upgrade task: Add decode check + download fallback before upgrade_plugin():

try:
    manager.decode_plugin_from_identifier(tenant_id, new_unique_identifier)
except Exception:
    pkg = download_plugin_pkg(new_unique_identifier)
    manager.upload_pkg(tenant_id, pkg, verify_signature=False)

manager.upgrade_plugin(...)

decode_plugin_from_identifier calls the daemon's GetPackage() which reads the actual file, so it raises an exception if the file is missing, triggering the re-download.

Start with PluginService.upgrade_plugin_with_marketplace() in api/services/plugin/plugin_service.py and process_tenant_plugin_autoupgrade_check_task() in api/tasks/process_tenant_plugin_autoupgrade_check_task.py. Trace how decode_plugin_from_identifier() checks the package and how download_plugin_pkg() and upload_pkg() are used. Done means both upgrade paths redownload a missing .difypkg file and complete the upgrade.