nextcloud / nextcloud/server

[Bug]: Folder sizes not always calculated with S3 primary storage

Open
#44,703 0 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

0. Needs triage 28-feedback bug feature: files feature: object storage
Dominant language
PHP
Stars
36.9k
Forks
5.2k
Avg merge
2d 3h
Merged PRs (30d)
713

Description

⚠️ This issue respects the following points: ⚠️
Bug description

When using S3 primary storage, some folders that have been uploaded via the web interface are shown with 0 KB size, despite the fact that they directly contain files with larger sizes.
This doesn't seem to happen with folders that were uploaded using the desktop client.

I can correct the folder sizes by deleting the folder, then going to the trash bin (where it now already shows the correct folder size), and restoring it.

Unfortunately, occ files:scan --all doesn't scan the S3 storage, so there's no way to fix the folder sizes with it. Cron jobs run regularly as well, but don't help.

Steps to reproduce
  1. Set up Nextcloud with S3 primary storage (Google Cloud Storage in my case)
  2. Upload a huge number of folders using the web interface
  3. Check if all folders got the correct folder size

Note that I'm not able to consistently reproduce this. I suspect it might have something to do with timeouts or retries during the upload.

Expected behavior

Even when using S3 primary storage, there should be a way to re-scan the files to fix the folder sizes, in case something goes wrong during the upload (which, arguably, shouldn't leave incorrect folder sizes, either).
occ files:scan --all doesn't scan the S3 primary storage.

It shouldn't be needed to delete the folders, then restore from the trash bin to fix the sizes.

Installation method

Community Docker image

Nextcloud Server version

28

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.2

Web server

Nginx

Database engine version

PostgreSQL

Is this bug present after an update or on a fresh install?

Fresh Nextcloud Server install

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?
  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO - SAML
  • Other
Configuration report
{
    "system": {
        "htaccess.RewriteBase": "\/",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "password": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "overwriteprotocol": "https",
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "objectstore": {
            "class": "\\OC\\Files\\ObjectStore\\S3",
            "arguments": {
                "bucket": "***REMOVED SENSITIVE VALUE***",
                "region": "europe-west1",
                "hostname": "storage.googleapis.com",
                "port": "80",
                "objectPrefix": "urn:oid:",
                "autocreate": false,
                "use_ssl": false,
                "use_path_style": false,
                "legacy_auth": false,
                "key": "***REMOVED SENSITIVE VALUE***",
                "secret": "***REMOVED SENSITIVE VALUE***"
            }
        },
        "filelocking.enabled": false,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***",
        ],
        "maintenance_window_start": 1,
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "trashbin_retention_obligation": "auto, 30",
        "versions_retention_obligation": "auto, 30",
        "dbtype": "pgsql",
        "version": "28.0.4.1",
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "465",
        "mail_smtpauth": 1,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpsecure": "ssl",
        "loglevel": 2,
        "maintenance": false
    }
}
List of activated Apps
Enabled:
  - activity: 2.20.0
  - cloud_federation_api: 1.11.0
  - contactsinteraction: 1.9.0
  - dav: 1.29.1
  - federatedfilesharing: 1.18.0
  - files: 2.0.0
  - files_pdfviewer: 2.9.0
  - files_sharing: 1.20.0
  - files_trashbin: 1.18.0
  - files_versions: 1.21.0
  - logreader: 2.13.0
  - lookup_server_connector: 1.16.0
  - nextcloud_announcements: 1.17.0
  - notifications: 2.16.0
  - oauth2: 1.16.3
  - password_policy: 1.18.0
  - privacy: 1.12.0
  - provisioning_api: 1.18.0
  - related_resources: 1.3.0
  - serverinfo: 1.18.0
  - settings: 1.10.1
  - sharebymail: 1.18.0
  - survey_client: 1.16.0
  - text: 3.9.1
  - theming: 2.3.0
  - twofactor_backupcodes: 1.17.0
  - updatenotification: 1.18.0
  - user_oidc: 5.0.2
  - viewer: 2.2.0
  - workflowengine: 2.10.0
Disabled:
  - admin_audit: 1.18.0
  - bruteforcesettings: 2.8.0
  - circles: 28.0.0 (installed 27.0.1)
  - comments: 1.18.0 (installed 1.17.0)
  - dashboard: 7.8.0 (installed 7.7.0)
  - encryption: 2.16.0
  - federation: 1.18.0 (installed 1.17.0)
  - files_external: 1.20.0 (installed 1.19.0)
  - files_reminders: 1.1.0 (installed 1.0.0)
  - firstrunwizard: 2.17.0 (installed 2.16.0)
  - photos: 2.4.0 (installed 2.3.0)
  - recommendations: 2.0.0 (installed 2.0.0)
  - support: 1.11.1 (installed 1.10.0)
  - suspicious_login: 6.0.0
  - systemtags: 1.18.0 (installed 1.17.0)
  - twofactor_totp: 10.0.0-beta.2
  - user_ldap: 1.19.0
  - user_status: 1.8.1 (installed 1.7.0)
  - weather_status: 1.8.0 (installed 1.7.0)
Nextcloud Signing status

No response

Nextcloud Logs

No response

Additional info

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the occ files:scan --all entry point and investigate how S3 primary storage is handled when folder sizes are updated after web uploads. Reproduce the zero-size behavior if possible, then verify that rescanning or the upload path leaves correct sizes without requiring deletion and restoration.

Written by the indexing model from the issue text.

Assessment

Tech stack
php
Domain
backend, cli, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.