abraunegg / abraunegg/onedrive
Modified-file upload: a timestamp-preserving local copy over a synced file is treated as "online is newer" and turned into a safeBackup + upload-as-new, although the online eTag is unchanged
- Langage dominant
- D
- Étoiles
- 12.8k
- Forks
- 911
- Merge moyen
- 1 j 8 h
- PR mergées (30 j)
- 25
Description
### Describe the bug
`uploadChangedLocalFileToOneDrive()` decides that the online file "already exists and is newer" by comparing the local mtime
with the online mtime (Issue #2626 Case 2-1 branch). The client has just fetched the online item and holds the eTag it recorded
at the last sync, but that eTag is not consulted. Any copy operation that preserves timestamps (macOS Finder copy, `cp -p`,
`rsync -t`, most backup tools) creates a **new** local file with an **old** date; when such a copy replaces a file the client
already synced, the local hash differs and the local mtime is older than the online one, so the branch fires, even though
**nothing has changed online** since the last sync (fresh online eTag == database eTag).
Result on master: the user's file is preserved as `--safeBackup-NNNN`, uploaded under that name, and the old online
bytes are downloaded over the real name. On 2.5.11 the local file is renamed to the safeBackup name, uploaded as new, the DB row
deleted and the old online file re-downloaded. Either way the user, who simply copied last month's workbook as this month's
template and renamed it, finds their work under a safeBackup name and last month's content under the real name. In an
accounting team that works "copy last period's folder/file, rename, edit" this happens daily; we had two incidents on two
consecutive days on one library.
The mtime comparison is the wrong instrument for "did online change since I last synced?". The eTag is the right one: an
online edit creates a new version and moves the eTag (verified: `{…},1 → {…},2` on a browser/API edit). Where the mtime test is
legitimately needed, a `--resync` run, where the database was just rebuilt from online and carries no information about what
this client last synced (#2626 Case 2-2), it must stay.
### Operating System Details
```shell
Linux ubuntu1 7.0.0-30-generic #30-Ubuntu SMP PREEMPT_DYNAMIC Fri Jul 31 18:22:54 UTC 2026 x86_64 GNU/Linux
Ubuntu 26.04.1 LTS
```
### Client Installation Method
From Source
### OneDrive Account Type
SharePoint
### What is your OneDrive Application Version
onedrive v2.5.11-41-g9b1b0d5 (master, reproduction); also v2.5.11 (production, same code path)
### What is your OneDrive Application Configuration
```shell
Defaults except `drive_id`, `sync_dir`, `skip_dir = "*.sb-*"`, `skip_dotfiles`, `check_nomount`, `classify_as_big_delete`,
`sync_dir_permissions = 750` / `sync_file_permissions = 640`, `threads = 4`, `disable_notifications`, `enable_logging`/`log_dir`.
No `--local-first`, no `--resync` (except control C2 below).
```
### What is your 'curl' version
```shell
curl 8.18.0 (x86_64-pc-linux-gnu) libcurl/8.18.0 OpenSSL/3.5.5 zlib/1.3.1 brotli/1.2.0 zstd/1.5.7 libidn2/2.3.8 libpsl/0.21.2 libssh2/1.11.1 nghttp2/1.68.0 librtmp/2.3 mit-krb5/1.22.1 OpenLDAP/2.6.10
```
### Where is your 'sync_dir' located
Local
### What are all your system 'mount points'
```shell
/dev/mapper/system-sharepoint on /srv/sharepoint type btrfs (rw,noatime,thread_pool=6,compress=zstd:1,ssd,space_cache=v2,subvolid=5,subvol=/)
```
### What are all your local file system partition types
```shell
btrfs (sync_dir), ext4 (system).
```
### How do you use 'onedrive'
One Linux server syncs eight SharePoint document libraries, one `--confdir` per library, each as a one-shot `--sync --verbose
--verbose` from a systemd timer every 15 minutes, as an unprivileged service user. The `sync_dir` is re-exported over Samba to
macOS clients, who copy/rename/edit Office files there. The bug reproduces with the client alone, on a single file, without the
share (steps below).
### Steps to reproduce the behaviour
Any valid `.xlsx` works; ours were 2–7 MB, both below and above the 4 MiB session threshold (the branch runs before the upload
method is chosen). `T1`, `T2` are two workbooks with different content; `T1` has mtime 60 days ago, `T2` 90 days ago.
1. `cp -p T1.xlsx /test/A.xlsx`; `onedrive --sync --verbose --verbose` → `Uploading new file … done`.
2. `rm A.xlsx; cp -p T2.xlsx A.xlsx` (new content, older timestamp, exactly what a Finder copy of last month's file does).
3. `onedrive --sync --verbose --verbose`.
4. Observe (excerpt below): `currentOnlineJSONData` eTag `"{…},1"` **equals** `database item` eTag `"{…},1"`, yet
`Skipping uploading this item as a locally modified file, will upload as a new file (online file already exists and is
newer)` → safeBackup → `Uploading new file: test/A--safeBackup-0001.xlsx … done` → `Downloading file: test/A.xlsx …
done` (old bytes back under the real name). 3 of 3 shapes (small file, large file, renamed-folder variant), on master and 2.5.11.
Controls (all behave the same with and without the change below): online really edited after the last sync (eTag differs) →
safeBackup as today (with `--local-first`; in normal order the delta pass handles it before this code runs); `--resync` →
safeBackup as today (Case 2-2, handled before this code runs); same content with another timestamp → timestamp corrected, no
upload; copy **without** `-p` (newer mtime) → modified upload.
### Complete Verbose Log Output
```shell
(Full `--verbose --verbose` logs of every scenario and control, for master with and without the change, sent by email as a 3.6 MB tar.gz.) Master `9b1b0d5`, step 3:
DEBUG: Local file time discrepancy detected: test/A.xlsx
DEBUG: The file content has changed locally and file now has a older timestamp. Uploading this file to OneDrive may potentially cause data-loss online
DEBUG: uploadChangedLocalFileToOneDrive: test/A.xlsx
DEBUG: currentOnlineJSONData: {…"@odata.etag":"\"{1150D9A8-…-70461D68D247},1\"","cTag":"\"c:{1150D9A8-…},2\""…"lastModifiedDateTime":"2026-09-09T17:37:33Z"…}
DEBUG: database item: Item("", "", "A.xlsx", "", file, "\"{1150D9A8-…-70461D68D247},1\"", "\"c:{1150D9A8-…},2\"", 2026-Sep-09 17:37:33Z, …)
DEBUG: Skipping uploading this item as a locally modified file, will upload as a new file (online file already exists and is newer): test/A.xlsx
DEBUG: The local item is out-of-sync with OneDrive, preserving to preserve existing file and prevent local data loss: test/A.xlsx -> test/A--safeBackup-0001.xlsx
DEBUG: Uploading new file: test/A--safeBackup-0001.xlsx ... done
DEBUG: Downloading file: test/A.xlsx ... done
DEBUG: Uploading modified file: test/A.xlsx ... failed!
Same file, same steps, master + the change below:
DEBUG: uploadChangedLocalFileToOneDrive: test/A.xlsx
DEBUG: Online eTag matches database eTag; treating as local modification despite older local timestamp: test/A.xlsx
DEBUG: Uploading modified file: test/A.xlsx ... done
(online eTag afterwards `"{…},2"`: one new version, previous content in version history.)
```
### Screenshots
_No response_
### Other Log Information or Details
```shell
```
### Additional context
**Proposed change** (tested, full diff at the end of this section, happy to open a PR): in `uploadChangedLocalFileToOneDrive()`, only take the "online
is newer" branch when the online item has actually changed since the last sync, or on a `--resync` run:
```d
bool onlineUnchangedSinceLastSync = hasETag(currentOnlineJSONData) && (currentETag == dbItem.eTag);
bool trustDatabaseBaseline = onlineUnchangedSinceLastSync && !appConfig.getValueBool("resync");
if ((localModifiedTime < onlineModifiedTime) && trustDatabaseBaseline) {
if (debugLogging) {addLogEntry("Online eTag matches database eTag; treating as local modification despite older local timestamp: " ~ localFilePath, ["debug"]);}
}
if ((localModifiedTime < onlineModifiedTime) && !trustDatabaseBaseline) {
// existing branch, unchanged
```
`hasETag()` is required so that the fallback `currentETag = dbItem.eTag` (online returned no eTag) cannot satisfy the test.
eTag rather than cTag: on SharePoint the eTag moves with every new version (content, rename, metadata), which makes it the
conservative choice; cTag additionally moves on SharePoint's own post-upload rewrite of Office files, which creates no version
and which the client already reconciles by download.
Related: #3289 fixed the neighbouring case (hash equal, only the timestamp differs). This is the hash-different / eTag-equal
case that was left as is.
**Regression evidence from production:** the guarded build has run on eight SharePoint libraries (one confdir each, 15-minute one-shot syncs) since 2026-09-09 19:28. On 2026-09-10, a heavy day on the busiest library (3,523 new uploads, 14 modified uploads, 678 downloads, 140 online deletes, mostly folder moves), the guard fired zero times and zero safeBackups were created, so it does not misfire on ordinary traffic; on 2026-09-08 and 2026-09-09, before the guard, the same library produced one incident a day of exactly this shape (a copy of last month's workbook renamed as this month's).
**The change against master `9b1b0d5`** (`git apply --check` clean):
```diff
diff --git a/src/sync.d b/src/sync.d
index b3d7cd0..4528ad1 100644
--- a/src/sync.d
+++ b/src/sync.d
@@ -8637,8 +8637,23 @@ class SyncEngine {
localModifiedTime.fracSecs = Duration.zero;
onlineModifiedTime.fracSecs = Duration.zero;
+ // An older local timestamp only means "online is newer" if the online item has actually changed since this
+ // client last synced it. Every copy operation that preserves timestamps (Finder copy, cp -p, rsync -t) produces
+ // a *new* local file carrying an *old* date, and the accounting workflow "copy last period's workbook as the
+ // template for this period" hits this on every use. If the eTag just fetched from online equals the eTag
+ // recorded in the database, nothing changed online: the local change is the only change and must be uploaded
+ // as a modification (versioning online keeps the previous content). eTag is used rather than cTag because it
+ // also moves on metadata-only changes, which keeps this the conservative choice.
+ // The timestamp comparison stays authoritative on a --resync run, where the database was just rebuilt from
+ // online and its eTag carries no information about what this client last synced (Issue #2626 Case 2-2).
+ bool onlineUnchangedSinceLastSync = hasETag(currentOnlineJSONData) && (currentETag == dbItem.eTag);
+ bool trustDatabaseBaseline = onlineUnchangedSinceLastSync && !appConfig.getValueBool("resync");
+ if ((localModifiedTime < onlineModifiedTime) && trustDatabaseBaseline) {
+ if (debugLogging) {addLogEntry("Online eTag matches database eTag; treating as local modification despite older local timestamp: " ~ localFilePath, ["debug"]);}
+ }
+
// Which file is newer? If local is newer, it will be uploaded as a modified file in the correct manner
- if (localModifiedTime < onlineModifiedTime) {
+ if ((localModifiedTime < onlineModifiedTime) && !trustDatabaseBaseline) {
// Online File is actually newer than the locally modified file
if (debugLogging) {
addLogEntry("currentOnlineJSONData: " ~ to!string(currentOnlineJSONData), ["debug"]);
```
Guide de contribution
Ouvrir le guide de contribution
Évaluation
Cette issue n'a pas encore été évaluée.