[Bug]: Large folder downloads delayed by HEAD request processing for authenticated users

⚠️ This issue respects the following points: ⚠️

• This is a bug, not a question or a configuration/webserver/proxy issue.
• This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
• Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
• I agree to follow Nextcloud's Code of Conduct.

Bug description

When downloading large files as an authenticated user via the webui, there is a significant delay before the download actually starts. The delay is proportional to the file count and total folder size. This only happens when downloading folders, single files download instantly. During the delay I can see high read disk io on the server.

However, when sharing the folder via a public link and downloading it from there starts instantly.

Using the browser console I see that when clicking download it first does a HEAD request (https://cloud.X.com/remote.php/dav/files/name/folder/?accept=zip ) first. During this time I can see a loading spinner and the dik io on the server, I do not receive any data to the client though. After a long time (e.g. a 70gb folder takes 6 minutes), the download starts. This time a GET request to the same url is sent and the expected download happens.

On the public link it does not send any HEAD request and just does the GET request and the download starts instantly.

Not sure if related but when I have debug logging enabled on the server I can see a lot of those errors (only when it sends the HEAD request):

Implicit conversion from float 1.8446744069414584E+19 to int loses precision at /var/www/html/3rdparty/deepdiver/zipstreamer/src/Count64.php#112
Implicit conversion from float 1.8446744069414584E+19 to int loses precision at /var/www/html/3rdparty/deepdiver/zipstreamer/src/Count64.php#119

The download does not fail however, it eventually starts and completes successfully.

Steps to reproduce

1. When logged in click download in the web interface for any folder
2. In the browser dev tools see a HEAD request being sent first which takes a long time
3. See high read disk io (presumably it's reading every file)
4. Eventually a GET request is sent after the HEAD request succeeds
5. Download starts in the browser

Expected behavior

The download starts instantly.

Nextcloud Server version

32

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.2

Web server

Nginx

Database engine version

PostgreSQL

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

• Default user-backend (database)
• LDAP/ Active Directory
• SSO - SAML
• Other

Configuration report

{
    "system": {
        "htaccess.RewriteBase": "\/",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": "0",
            "password": "***REMOVED SENSITIVE VALUE***"
        },
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "cloud.x.com"
        ],
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "default_language": "de",
        "default_phone_region": "ch",
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "pgsql",
        "version": "32.0.4.1",
        "overwrite.cli.url": "https:\/\/cloud.x.com",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "6432",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "465",
        "overwritehost": "cloud.x.com",
        "overwriteprotocol": "https",
        "loglevel": 3,
        "log_type": "syslog",
        "syslog_tag": "Nextcloud",
        "logfile": "",
        "maintenance": false,
        "maintenance_window_start": 1,
        "defaultapp": "",
        "allow_local_remote_servers": true,
        "simpleSignUpLink.shown": false,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "enabledPreviewProviders": [
            "OC\\Preview\\MP3",
            "OC\\Preview\\TXT",
            "OC\\Preview\\MarkDown",
            "OC\\Preview\\OpenDocument",
            "OC\\Preview\\Krita",
            "OC\\Preview\\Imaginary",
            "OC\\Preview\\Movie"
        ],
        "preview_imaginary_url": "***REMOVED SENSITIVE VALUE***",
        "allow_user_to_change_display_name": false,
        "lost_password_link": "disabled",
        "app_install_overwrite": {
            "1": "htmlviewer"
        },
        "onlyoffice": {
            "editors_check_interval": "0"
        },
        "auth.webauthn.enabled": false,
        "upgrade.disable-web": false,
        "theme": "",
        "user_oidc": {
            "enrich_login_id_token_with_userinfo": true
        },
        "session_lifetime": 86400,
        "session_keepalive": true,
        "session_relaxed_expiry": true,
        "remember_login_cookie_lifetime": 1296000,
        "mail_smtpauth": true,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpsecure": "ssl",
        "dbpersistent": true,
        "tempdirectory": "\/mnt\/nextcloud-tmp"
    }
}

List of activated Apps

Enabled:
  - activity: 5.0.0-dev.0
  - admin_audit: 1.22.0
  - app_api: 32.0.0
  - appointments: 2.6.2
  - bruteforcesettings: 5.0.0-dev.0
  - calendar: 6.1.5
  - circles: 32.0.0
  - cloud_federation_api: 1.16.0
  - comments: 1.22.0
  - contacts: 8.3.0
  - contactsinteraction: 1.13.1
  - dashboard: 7.12.0
  - dav: 1.34.2
  - deck: 1.16.3
  - end_to_end_encryption: 1.18.0
  - external: 7.0.0
  - federatedfilesharing: 1.22.0
  - files: 2.4.0
  - files_3dmodelviewer: 0.0.16
  - files_downloadlimit: 5.0.0-dev.0
  - files_fulltextsearch: 32.0.2
  - files_linkeditor: 1.1.23
  - files_lock: 32.0.1
  - files_pdfviewer: 5.0.0-dev.0
  - files_reminders: 1.5.0
  - files_sharing: 1.24.1
  - files_trashbin: 1.22.0
  - files_versions: 1.25.0
  - fileslibreofficeedit: 2.0.1
  - forms: 5.2.4
  - fulltextsearch: 32.0.0
  - fulltextsearch_elasticsearch: 32.0.2
  - groupfolders: 20.1.9
  - logreader: 5.0.0-dev.0
  - lookup_server_connector: 1.20.0
  - mail: 5.6.8
  - metadata: 0.23.0
  - music: 2.5.1
  - news: 27.2.0
  - notes: 4.12.4
  - notifications: 5.0.0-dev.0
  - notify_push: 1.3.0
  - oauth2: 1.20.0
  - photos: 5.0.0-dev.1
  - privacy: 4.0.0-dev.0
  - profile: 1.1.0
  - provisioning_api: 1.22.0
  - recommendations: 5.0.0-dev.0
  - related_resources: 3.0.0-dev.0
  - richdocuments: 9.0.2
  - serverinfo: 4.0.0-dev.0
  - settings: 1.15.1
  - sharebymail: 1.22.0
  - socialsharing_email: 4.0.1
  - spreed: 22.0.8
  - suspicious_login: 10.0.0-dev.0
  - tasks: 0.17.1
  - text: 6.0.1
  - theming: 2.7.0
  - thesearchpage: 2.0.8
  - twofactor_backupcodes: 1.21.0
  - updatenotification: 1.22.0
  - user_oidc: 8.3.0
  - user_status: 1.12.0
  - viewer: 5.0.0-dev.0
  - weather_status: 1.12.0
  - whiteboard: 1.5.3
  - workflow_pdf_converter: 3.0.0
  - workflowengine: 2.14.0
Disabled:
  - announcementcenter: 7.3.0 (installed 7.3.0)
  - checksum: 2.0.3 (installed 2.0.3)
  - cookbook: 0.11.5 (installed 0.11.5)
  - emlviewer: 2.0.1 (installed 2.0.1)
  - encryption: 2.20.0
  - epubviewer: 1.8.1 (installed 1.8.1)
  - federation: 1.22.0 (installed 1.20.0)
  - files_automatedtagging: 3.0.3 (installed 3.0.3)
  - files_external: 1.24.0
  - firstrunwizard: 5.0.0-dev.0 (installed 3.0.0)
  - flow_notifications: 3.0.0 (installed 3.0.0)
  - htmlviewer: 32.0.0 (installed 32.0.0)
  - imageconverter: 2.1.0 (installed 2.1.0)
  - nextcloud_announcements: 4.0.0-dev.0 (installed 1.18.0)
  - password_policy: 4.0.0-dev.0 (installed 2.0.0)
  - support: 4.0.0-dev.0 (installed 4.0.0-dev.0)
  - survey_client: 4.0.0-dev.0 (installed 1.16.0)
  - systemtags: 1.22.0 (installed 1.21.1)
  - talk_matterbridge: 1.33.1026000 (installed 1.33.1026000)
  - twofactor_nextcloud_notification: 6.0.0-dev.0
  - twofactor_totp: 14.0.0 (installed 12.0.0-dev)
  - user_ldap: 1.23.0
  - webhook_listeners: 1.3.0 (installed 1.1.0-dev)
  - workflow_ocr: 1.32.1 (installed 1.32.1)
  - workflow_script: 3.0.0 (installed 3.0.0)

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

Additional info

When I looked at the server-side code, I found that SabreDAV CorePlugin.php converts HEAD requests to GET requests internally. It does that around line 242 in the httpHead() function. It seems to clone the request and change the method to GET while adding a header X-Sabre-Original-Method: HEAD. (line 248)

The standard CorePlugin::httpGet() function (line 73) checks for this header and skips body creation if it detects a HEAD request.

However in the ZipFolderPlugin.php, I didn't see any similar check for the X-Sabre-Original-Metod header. The handleDownload (line 97) function seems to process HEAD requests the same way as GET requests, which might explain why it's taking so long and I only see read io on the server (like it would when downloading).

I'm not a php developer, so these are just assumptions based on what I observed.

Start with CorePlugin.php around httpHead() and httpGet(), then inspect ZipFolderPlugin.php and its handleDownload() path for the X-Sabre-Original-Method header. Reproduce the authenticated folder download and compare HEAD and GET behavior. Done means the HEAD request no longer performs the full folder archive work, so the subsequent download starts without the reported delay.