nextcloud / nextcloud/integration_onedrive
A single interrupted batch ends the files import for good, while the UI keeps showing "importing"
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 41
- Forks
- 12
- Avg merge
- 6d 16h
- Merged PRs (30d)
- 4
Description
How to use GitHub
- Please use the 👍 reaction to show that you are affected by the same issue.
- Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
- Subscribe to receive notifications on status change and new comments.
Steps to reproduce
- Connect a OneDrive account and start a files import large enough to need many batches.
- While a batch is running, interrupt the PHP process — restart the cron service, reboot, deploy, kill a wedged download, or let it be OOM-killed. In my case a routine service restart during a system upgrade was enough: the unit stops
cron.phpwith SIGTERM while the import job is running. - Check the job list:
$ occ background-job:list -c 'OCA\Onedrive\BackgroundJob\ImportOnedriveJob'
(no output)
Nothing ever resumes, and the Data migration page still shows the import as in progress.
Expected behaviour
An interrupted batch is retried on a later cron run and the import continues on its own. If it truly cannot continue, the UI should stop reporting an import in progress, so the user can restart it from the interface.
Actual behaviour
The import stops permanently and silently. The UI keeps claiming it is running, so there is no indication anything is wrong, and the user cannot restart it from the interface either.
The cause is a gap between two points in the job lifecycle:
QueuedJob::start()(server) removes the job'soc_jobsrow beforerun()executes.OnedriveStorageAPIService::importOnedriveJob()re-adds the job with$this->jobList->add(ImportOnedriveJob::class, ['user_id' => $userId])only at the end of a successful batch.
Any exit between those two points — signal, fatal, OOM — leaves no job row at all, while the importing_onedrive user value is still '1'.
Recovery is not possible from the UI, because startImportOnedrive() returns early while the flag is set:
$alreadyImporting = $this->config->getUserValue($userId, Application::APP_ID, 'importing_onedrive', '0') === '1';
if ($alreadyImporting) {
return ['targetPath' => $targetPath];
}
so the import button silently does nothing. The user has to clear the flag out of band first, e.g.
occ user:setting <uid> integration_onedrive importing_onedrive 0
and only then start the import again.
The app already anticipates stale runs — importOnedriveJob() checks onedrive_import_running against Application::IMPORT_JOB_TIMEOUT — but that guard only helps if a job still exists to run it, which after an interruption it does not.
Secondary, same code path: startImportOnedrive() resets imported_size and last_onedrive_import_timestamp and deletes import_tree, but does not reset nb_imported_files. After a restart the file counter carries on from its old value while the byte counter restarts at zero, so the UI shows something like "2195 files imported (609.2 MiB)" when only 2 of those files were downloaded in the current run. Both counters should be reset together.
Note on restarting: because import_tree is deleted, a restart re-walks the whole drive over the Graph API. Files already present are correctly skipped by getFile() (nodeExists() → return 0), so nothing is re-downloaded, but on a large drive the re-listing is slow and costs a request per folder.
Suggested fixes
- Re-add the job on failure — a
finally, or catch-and-requeue — so an interrupted batch resumes on the next cron run. - Or add a small
TimedJobthat re-queuesImportOnedriveJobwheneverimporting_onedrive === '1'and no job is queued for that user. This also covers the case where the process had to be killed. - Reset
nb_imported_filesalongsideimported_sizeinstartImportOnedrive().
Server configuration
Web server: Nginx (behind Traefik)
Database: PostgreSQL 17.10
PHP version: 8.4.23
Nextcloud version: 34.0.2
integration_onedrive version: 3.5.2
List of activated apps
Enabled:
- activity: 7.0.0
- app_api: 34.0.0
- appstore: 1.0.0
- bruteforcesettings: 7.0.0
- calendar: 6.5.2
- circles: 34.0.0
- cloud_federation_api: 1.18.0
- comments: 1.24.0
- contacts: 8.7.5
- contactsinteraction: 1.15.0
- dashboard: 7.14.0
- dav: 1.39.0
- deck: 1.18.3
- federatedfilesharing: 1.24.0
- federation: 1.24.0
- files: 2.6.0
- files_downloadlimit: 5.2.0
- files_lock: 34.0.1
- files_pdfviewer: 7.0.0-dev.0
- files_reminders: 1.7.0
- files_sharing: 1.26.0
- files_trashbin: 1.24.0
- files_versions: 1.27.0
- firstrunwizard: 7.0.0-dev.0
- forms: 5.3.5
- groupfolders: 22.0.6
- integration_giphy: 2.2.1
- integration_onedrive: 3.5.2
- logreader: 7.0.0
- lookup_server_connector: 1.22.0
- mail: 5.10.10
- maps: 1.7.1
- music: 3.1.1
- nextcloud_announcements: 6.0.0
- notes: 6.0.1
- notifications: 7.0.0-dev.1
- notify_push: 1.3.3
- oauth2: 1.22.0
- office: 1.0.0
- password_policy: 6.0.0-dev.0
- photos: 7.0.0
- privacy: 6.0.0-dev.1
- profile: 1.3.0
- provisioning_api: 1.24.0
- recommendations: 7.0.0
- related_resources: 5.0.0-dev.0
- serverinfo: 6.0.0
- settings: 1.17.0
- sharebymail: 1.24.0
- spreed: 24.0.3
- support: 6.0.0
- survey_client: 6.0.0-dev.0
- systemtags: 1.24.0
- tasks: 0.18.1
- text: 8.0.0
- theming: 2.9.0
- twofactor_backupcodes: 1.23.0
- twofactor_totp: 16.0.0
- twofactor_webauthn: 2.7.0
- updatenotification: 1.24.0
- user_status: 1.14.0
- viewer: 7.0.0-dev.0
- weather_status: 1.14.0
- webhook_listeners: 1.6.0
- workflowengine: 2.16.0
Disabled:
- admin_audit: 1.24.0
- encryption: 2.22.0
- files_external: 1.26.0
- onlyoffice: 10.1.2 (installed 10.1.2)
- suspicious_login: 12.0.0-dev.0
- twofactor_nextcloud_notification: 8.0.0
- user_ldap: 1.25.0
Nextcloud configuration
```json { "system": { "apps_paths": [ { "path": "\/nix\/store\/pppzqxmrmg6rjx387c2sp1cmqzmd00gx-nextcloud-34.0.2-with-apps\/apps", "url": "\/apps", "writable": false }, { "path": "\/nix\/store\/pppzqxmrmg6rjx387c2sp1cmqzmd00gx-nextcloud-34.0.2-with-apps\/nix-apps", "url": "\/nix-apps", "writable": false }, { "path": "\/nix\/store\/pppzqxmrmg6rjx387c2sp1cmqzmd00gx-nextcloud-34.0.2-with-apps\/store-apps", "url": "\/store-apps", "writable": true } ], "memcache.local": "\\OC\\Memcache\\APCu", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbuser": "***REMOVED SENSITIVE VALUE***", "dbtype": "pgsql", "datadirectory": "***REMOVED SENSITIVE VALUE***", "default_certificates_bundle_path": "\/nix\/store\/30wqfg7d7nxlqmlnxad14s2r775bq5mw-nss-cacert-3.125\/etc\/ssl\/certs\/ca-bundle.crt", "default_phone_region": "BR", "enabledPreviewProviders": [ "OC\\Preview\\PDF", "OC\\Preview\\SVG", "OC\\Preview\\BMP", "OC\\Preview\\GIF", "OC\\Preview\\JPEG", "OC\\Preview\\Krita", "OC\\Preview\\MarkDown", "OC\\Preview\\MP3", "OC\\Preview\\OpenDocument", "OC\\Preview\\PNG", "OC\\Preview\\TXT", "OC\\Preview\\XBitmap", "OC\\Preview\\HEIC" ], "forwarded_for_headers": [ "HTTP_X_FORWARDED_FOR", "HTTP_CF_CONNECTING_IP" ], "integrity.check.disabled": true, "log_type": "file", "loglevel": 1, "mail_domain": "***REMOVED SENSITIVE VALUE***", "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_send_plaintext_only": false, "mail_sendmailmode": "smtp", "mail_smtpauth": true, "mail_smtpdebug": false, "mail_smtphost": "***REMOVED SENSITIVE VALUE***", "mail_smtpmode": "smtp", "mail_smtpname": "***REMOVED SENSITIVE VALUE***", "mail_smtpport": 465, "mail_smtpsecure": "ssl", "mail_smtpstreamoptions": [], "mail_smtptimeout": 10, "mail_template_class": "\\OC\\Mail\\EMailTemplate", "maintenance_window_start": 5, "maxUploadSize": "512M", "memcache.distributed": "\\OC\\Memcache\\Redis", "memcache.locking": "\\OC\\Memcache\\Redis", "overwrite.cli.url": "REDACTED", "overwriteprotocol": "", "profile.enabled": false, "redis": { "host": "***REMOVED SENSITIVE VALUE***", "port": 0 }, "skeletondirectory": "", "trusted_proxies": "***REMOVED SENSITIVE VALUE***", "upgrade.disable-web": true, "instanceid": "***REMOVED SENSITIVE VALUE***", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "mail_smtppassword": "***REMOVED SENSITIVE VALUE***", "version": "34.0.2.1", "dbtableprefix": "oc_", "dbpassword": "***REMOVED SENSITIVE VALUE***", "installed": true, "twofactor_enforced": "true", "twofactor_enforced_groups": [], "twofactor_enforced_excluded_groups": [], "maintenance": false, "app_install_overwrite": [], "appstoreenabled": true, "allow_local_remote_servers": true, "logfile": "\/var\/log\/nextcloud\/nextcloud.log", "trusted_domains": [ "REDACTED", "REDACTED" ], "PASS": "***REMOVED SENSITIVE VALUE***" } } ```Browser
N/A — server-side background job behaviour. The only UI symptom is the Data migration page permanently showing an import in progress, which follows from the server state described above.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with QueuedJob::start(), OnedriveStorageAPIService::importOnedriveJob(), and startImportOnedrive(), following how ImportOnedriveJob is queued and how the importing_onedrive state is updated. Reproduce an interrupted batch if possible, then ensure the job can resume or the UI can restart it, and that imported_size and nb_imported_files are reset together.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100