Files import hangs for hours on an expired downloadUrl — no read timeout, and the failed file is then dropped for good

How to use GitHub

• Please use the 👍 reaction to show that you are affected by the same issue.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

Steps to reproduce

1. Connect a OneDrive account and start a files import of a large drive (thousands of files, several GB), so the import runs as a chain of ImportOnedriveJob batches under system cron.
2. Let it run long enough that a folder listing is older than the lifetime of the @microsoft.graph.downloadUrl values it returned (about an hour here).
3. Observe the cron process: it stops making progress and never finishes. Nothing is logged.

Expected behaviour

1. The download URL used for a file should still be valid when the file is fetched.
2. A transfer that stalls should hit a read timeout and raise, instead of blocking forever.
3. A file that fails to download should be retried, or at least reported to the user at the end of the import.

Actual behaviour

The background job blocks indefinitely with no timeout, no error, no log line and no heartbeat. The cron unit simply stays active for hours.

Concretely, on my instance (import of ~2000 files; local times are -03, log times UTC):

• 20:48 — cron run starts, the import batch progresses normally for hours.
• 00:25 (03:25 UTC) — the job opens .../uploads/dnu_old/DSC02561.jpg for writing and stops.
• 00:25 → 00:40+ — frozen: the unit's IPIngressBytes does not move, /proc/<pid>/fdinfo/<fd> pos: does not advance, imported_size / nb_imported_files / last_onedrive_import_timestamp are unchanged, and no file under the data directory has been touched.

$ sudo ls -l /proc/$pid/fd | grep -v 'socket:\|pipe:\|/dev/\|anon_inode'
lr-x------ 1 nextcloud nextcloud 64 Jul 31 00:25 10 -> /var/lib/nextcloud/data/<user>/files/.../dnu_old
lr-x------ 1 nextcloud nextcloud 64 Jul 30 20:48 3  -> /nix/store/…-nextcloud-34.0.2/cron.php
l-wx------ 1 nextcloud nextcloud 64 Jul 31 00:25 6  -> /var/lib/nextcloud/data/<user>/files/.../dnu_old/DSC02561.jpg

It is not a large-file problem — the stalled item is an ordinary JPEG.

The URL had already expired when it was used. getFile() downloads from $fileItem['@microsoft.graph.downloadUrl'], which comes from the folder listing. That URL carries a short-lived tempauth token; decoding the one from this failure gives "exp": "1785463837" = 2026-07-31 02:10:37 UTC, while the download was started at 03:25 UTC and finally errored at 03:49 UTC — roughly 75 minutes after expiry. On a large import, the loop reaches file N long after the listing that produced its URL.

Instead of a clean 401 — which fileRequest() would handle via $respCode >= 400 → 'Bad credentials' — the endpoint accepted the connection and then sent nothing at all. Combined with the Guzzle options in lib/Service/OnedriveAPIService.php::fileRequest():

$options = [
    'sink' => $resource,
    'timeout' => 0,
    'headers' => [ 'User-Agent' => $this->userAgent ],
    'handler' => $this->getKiotaHandlerStack(),
];
$response = $this->client->get($url, $options);

'timeout' => 0 disables the transfer timeout and no read_timeout is set, so the job blocks in read() with no bound.

Everything downstream of the exception is already correct. Killing the TCP connection by hand proves it:

sudo ss -K "dst <peer-ip> and dport = 443"

The request raises, fileRequest() returns ['error' => …], getFile() logs the warning, deletes the partial file, returns null, and the import resumes on its own — no restart, no lost job:

{"time":"2026-07-31T03:49:46+00:00","level":2,"app":"integration_onedrive","user":"<user>",
 "message":"OneDrive error downloading file DSC02561.jpg : cURL error 56: Recv failure: Software caused connection abort (…) for https://my.microsoftpersonalcontent.com/personal/REDACTED/_layouts/15/download.aspx?UniqueId=REDACTED&Translate=false&tempauth=REDACTED&ApiVersion=2.0",
 "version":"34.0.2.1"}

So only the two guards in front of it are missing.

Two further problems visible in that same log line:

The warning logs the full download URL, including the tempauth bearer token (and the siteid/appid encoded in it). Anyone attaching a log excerpt to a bug report — as this template asks them to — publishes a credential for their own drive. I had to redact the line above by hand. The message should carry the file name and the cURL error, not the URL.

A file that fails to download is dropped permanently. getFile() returns null, the loop continues, import_tree marks the folder done, and there is no retry list and no failed-file record; the completion notification reports totalSeenNumber regardless. In this incident two files (DSC02561.jpg and, as collateral of the connection kill, DSC02561_thumb.jpg) are now silently missing from the import and nothing will ever fetch them. For a data-migration tool, a transient network error causing unreported missing files is worse than the hang itself.

Suggested fixes

1. Don't reuse a cached @microsoft.graph.downloadUrl of unknown age — re-resolve per file (GET /me/drive/items/{id}/content returns a redirect to a fresh URL), or refuse to use one older than a few minutes.
2. Set a finite read_timeout in fileRequest() as a backstop, so any silent far end costs seconds rather than hours. read_timeout rather than timeout, since the latter bounds the whole transfer and would break legitimately long downloads of large files.
3. Log the file name and error, not the URL.
4. Retry failed downloads, or collect them and surface them to the user when the import finishes.

Related: #138 — because QueuedJob::start() deletes the job's oc_jobs row before run(), a hung import is externally indistinguishable from one that has already finished, which is what makes this so hard to diagnose.

Server configuration

Web server: Nginx (behind Traefik)

Database: PostgreSQL 17.10

PHP version: 8.4.23

Nextcloud version: 34.0.2

integration_onedrive version: 3.5.2

Enabled:
  - activity: 7.0.0
  - app_api: 34.0.0
  - appstore: 1.0.0
  - bruteforcesettings: 7.0.0
  - calendar: 6.5.2
  - circles: 34.0.0
  - cloud_federation_api: 1.18.0
  - comments: 1.24.0
  - contacts: 8.7.5
  - contactsinteraction: 1.15.0
  - dashboard: 7.14.0
  - dav: 1.39.0
  - deck: 1.18.3
  - federatedfilesharing: 1.24.0
  - federation: 1.24.0
  - files: 2.6.0
  - files_downloadlimit: 5.2.0
  - files_lock: 34.0.1
  - files_pdfviewer: 7.0.0-dev.0
  - files_reminders: 1.7.0
  - files_sharing: 1.26.0
  - files_trashbin: 1.24.0
  - files_versions: 1.27.0
  - firstrunwizard: 7.0.0-dev.0
  - forms: 5.3.5
  - groupfolders: 22.0.6
  - integration_giphy: 2.2.1
  - integration_onedrive: 3.5.2
  - logreader: 7.0.0
  - lookup_server_connector: 1.22.0
  - mail: 5.10.10
  - maps: 1.7.1
  - music: 3.1.1
  - nextcloud_announcements: 6.0.0
  - notes: 6.0.1
  - notifications: 7.0.0-dev.1
  - notify_push: 1.3.3
  - oauth2: 1.22.0
  - office: 1.0.0
  - password_policy: 6.0.0-dev.0
  - photos: 7.0.0
  - privacy: 6.0.0-dev.1
  - profile: 1.3.0
  - provisioning_api: 1.24.0
  - recommendations: 7.0.0
  - related_resources: 5.0.0-dev.0
  - serverinfo: 6.0.0
  - settings: 1.17.0
  - sharebymail: 1.24.0
  - spreed: 24.0.3
  - support: 6.0.0
  - survey_client: 6.0.0-dev.0
  - systemtags: 1.24.0
  - tasks: 0.18.1
  - text: 8.0.0
  - theming: 2.9.0
  - twofactor_backupcodes: 1.23.0
  - twofactor_totp: 16.0.0
  - twofactor_webauthn: 2.7.0
  - updatenotification: 1.24.0
  - user_status: 1.14.0
  - viewer: 7.0.0-dev.0
  - weather_status: 1.14.0
  - webhook_listeners: 1.6.0
  - workflowengine: 2.16.0
Disabled:
  - admin_audit: 1.24.0
  - encryption: 2.22.0
  - files_external: 1.26.0
  - onlyoffice: 10.1.2 (installed 10.1.2)
  - suspicious_login: 12.0.0-dev.0
  - twofactor_nextcloud_notification: 8.0.0
  - user_ldap: 1.25.0

{
    "system": {
        "apps_paths": [
            {
                "path": "\/nix\/store\/pppzqxmrmg6rjx387c2sp1cmqzmd00gx-nextcloud-34.0.2-with-apps\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/nix\/store\/pppzqxmrmg6rjx387c2sp1cmqzmd00gx-nextcloud-34.0.2-with-apps\/nix-apps",
                "url": "\/nix-apps",
                "writable": false
            },
            {
                "path": "\/nix\/store\/pppzqxmrmg6rjx387c2sp1cmqzmd00gx-nextcloud-34.0.2-with-apps\/store-apps",
                "url": "\/store-apps",
                "writable": true
            }
        ],
        "memcache.local": "\\OC\\Memcache\\APCu",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "pgsql",
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "default_certificates_bundle_path": "\/nix\/store\/30wqfg7d7nxlqmlnxad14s2r775bq5mw-nss-cacert-3.125\/etc\/ssl\/certs\/ca-bundle.crt",
        "default_phone_region": "BR",
        "enabledPreviewProviders": [
            "OC\\Preview\\PDF",
            "OC\\Preview\\SVG",
            "OC\\Preview\\BMP",
            "OC\\Preview\\GIF",
            "OC\\Preview\\JPEG",
            "OC\\Preview\\Krita",
            "OC\\Preview\\MarkDown",
            "OC\\Preview\\MP3",
            "OC\\Preview\\OpenDocument",
            "OC\\Preview\\PNG",
            "OC\\Preview\\TXT",
            "OC\\Preview\\XBitmap",
            "OC\\Preview\\HEIC"
        ],
        "forwarded_for_headers": [
            "HTTP_X_FORWARDED_FOR",
            "HTTP_CF_CONNECTING_IP"
        ],
        "integrity.check.disabled": true,
        "log_type": "file",
        "loglevel": 1,
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_send_plaintext_only": false,
        "mail_sendmailmode": "smtp",
        "mail_smtpauth": true,
        "mail_smtpdebug": false,
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "smtp",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": 465,
        "mail_smtpsecure": "ssl",
        "mail_smtpstreamoptions": [],
        "mail_smtptimeout": 10,
        "mail_template_class": "\\OC\\Mail\\EMailTemplate",
        "maintenance_window_start": 5,
        "maxUploadSize": "512M",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "overwrite.cli.url": "REDACTED",
        "overwriteprotocol": "",
        "profile.enabled": false,
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0
        },
        "skeletondirectory": "",
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "upgrade.disable-web": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "version": "34.0.2.1",
        "dbtableprefix": "oc_",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "twofactor_enforced": "true",
        "twofactor_enforced_groups": [],
        "twofactor_enforced_excluded_groups": [],
        "maintenance": false,
        "app_install_overwrite": [],
        "appstoreenabled": true,
        "allow_local_remote_servers": true,
        "logfile": "\/var\/log\/nextcloud\/nextcloud.log",
        "trusted_domains": [
            "REDACTED",
            "REDACTED"
        ],
        "PASS": "***REMOVED SENSITIVE VALUE***"
    }
}

Browser

N/A — this is a server-side background job, no browser involved.

Start with lib/Service/OnedriveAPIService.php::fileRequest() and trace how getFile(), import_tree, and ImportOnedriveJob handle download failures. Verify the cached download URL flow and existing error logging before deciding how refreshes, read timeouts, retries, and failed-file reporting fit together. Done means stalled or expired downloads cannot block the import, URLs are not exposed in logs, and failed files are retried or surfaced to the user.