nextcloud / nextcloud/server

S3 external storage created via `occ files_external:create` does not detect remote file changes

Open
#53,249 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

1. to develop 30-feedback bug feature: external storage feature: object storage feature: occ good first issue papercut
Dominant language
PHP
Stars
36.9k
Forks
5.2k
Avg merge
2d 3h
Merged PRs (30d)
713

Description

⚠️ This issue respects the following points: ⚠️
Bug description

Hi,

Issue:
Nextcloud is supposed to follow any change in S3 directory on access, as configured:

Image
However, when I add/delete a file on S3 side, Nextcloud does not update when I access the directory containing the file. Only if I do a manual scan occ files:scan or if I try to access the file in Nextcloud, which results in nothing, then after a browser refresh the list is correct.

Workaround:
I added a file scan every 5 min in cron, but this may not be good for S3 server to be interrogated for nothing, and we have a 5min delay.

Steps to reproduce
  1. deploy Nextcloud
  2. deploy external storage app
  3. configure S3 remote
  4. add file in S3
  5. see if the file appear in Nextcloud immediately
    => the file is not present
Expected behavior

Any access on Nextcloud will show correct info of file in remote.

Nextcloud Server version

30

Operating system

Other

PHP engine version

PHP 8.2

Web server

Nginx

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Fresh Nextcloud Server install

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?
  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO - SAML
  • Other
Configuration report
{
    "system": {
        "htaccess.RewriteBase": "\/",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "debug": true,
        "check_data_directory_permissions": false,
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "forwarded_for_headers": [
            "HTTP_X_FORWARDED_FOR"
        ],
        "skeletondirectory": "\/dev\/null",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "password": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "upgrade.disable-web": true,
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": "***REMOVED SENSITIVE VALUE***"
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "30.0.6.2",
        "overwrite.cli.url": "http:\/\/localhost",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
        "filelocking.enabled": false
    }
}
List of activated Apps
Enabled:
  - activity: 3.0.0
  - bruteforcesettings: 3.0.0
  - cloud_federation_api: 1.13.0
  - comments: 1.20.1
  - dav: 1.31.1
  - federatedfilesharing: 1.20.0
  - files: 2.2.0
  - files_downloadlimit: 3.0.0
  - files_external: 1.22.0
  - files_pdfviewer: 3.0.0
  - files_sharing: 1.22.0
  - logreader: 3.0.0
  - lookup_server_connector: 1.18.0
  - notifications: 3.0.0
  - oauth2: 1.18.1
  - password_policy: 2.0.0
  - provisioning_api: 1.20.0
  - serverinfo: 2.0.0
  - settings: 1.13.0
  - support: 2.0.0
  - survey_client: 2.0.0
  - systemtags: 1.20.0
  - text: 4.1.0
  - theming: 2.6.0
  - twofactor_backupcodes: 1.19.0
  - updatenotification: 1.20.0
  - user_ldap: 1.21.0
  - viewer: 3.0.0
  - workflowengine: 2.12.0
Enabled:
  - activity: 3.0.0
  - bruteforcesettings: 3.0.0
  - cloud_federation_api: 1.13.0
  - comments: 1.20.1
  - dav: 1.31.1
  - federatedfilesharing: 1.20.0
  - files: 2.2.0
  - files_downloadlimit: 3.0.0
  - files_external: 1.22.0
  - files_pdfviewer: 3.0.0
  - files_sharing: 1.22.0
  - logreader: 3.0.0
  - lookup_server_connector: 1.18.0
  - notifications: 3.0.0
  - oauth2: 1.18.1
  - password_policy: 2.0.0
  - provisioning_api: 1.20.0
  - serverinfo: 2.0.0
  - settings: 1.13.0
  - support: 2.0.0
  - survey_client: 2.0.0
  - systemtags: 1.20.0
  - text: 4.1.0
  - theming: 2.6.0
  - twofactor_backupcodes: 1.19.0
  - updatenotification: 1.20.0
  - user_ldap: 1.21.0
  - viewer: 3.0.0
  - workflowengine: 2.12.0Disabled:
  - admin_audit: 1.20.0
  - app_api: 4.0.6 (installed 4.0.6)
  - circles: 30.0.0 (installed 30.0.0)
  - contactsinteraction: 1.11.0 (installed 1.11.0)
  - dashboard: 7.10.0 (installed 7.10.0)
  - encryption: 2.18.0
  - federation: 1.20.0 (installed 1.20.0)
  - files_reminders: 1.3.0 (installed 1.3.0)
  - files_trashbin: 1.20.1 (installed 1.20.1)
  - files_versions: 1.23.0 (installed 1.23.0)
  - firstrunwizard: 3.0.0 (installed 3.0.0)
  - nextcloud_announcements: 2.0.0 (installed 2.0.0)
  - photos: 3.0.2 (installed 3.0.2)
  - privacy: 2.0.0 (installed 2.0.0)
  - recommendations: 3.0.0 (installed 3.0.0)
  - related_resources: 1.5.0 (installed 1.5.0)
  - sharebymail: 1.20.0 (installed 1.20.0)
  - suspicious_login: 8.0.0
  - twofactor_nextcloud_notification: 4.0.0
  - twofactor_totp: 12.0.0-dev
  - user_status: 1.10.0 (installed 1.10.0)
  - weather_status: 1.10.0 (installed 1.10.0)
  - webhook_listeners: 1.1.0-dev (installed 1.1.0-dev)
Nextcloud Signing status
No errors have been found.
Nextcloud Logs

Additional info

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the files_external app flow used by occ files_external:create when an S3 directory is accessed, and compare it with the behavior after occ files:scan. Reproduce the external file add/delete sequence from the issue and inspect the related storage listing and cache paths. Done means directory access reflects remote S3 changes without requiring a manual scan or opening the missing file.

Written by the indexing model from the issue text.

Assessment

Tech stack
php
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.