[Note / no action required] What it would take to move away from MATLAB datenums for timebases/timestamps

This is an informational design note, not a bug or a scheduled task. It records, for future reference, every place MATLAB datenum is currently baked into the DID/NDI/NDR stack and what a coordinated migration would involve, so the decision can be made deliberately later. No resolution is required; the current datenum-based behavior is intentional and backward-compatible. Decision (2026-06): keep datenum for now.

Context: came out of reviewing DID-matlab #146 (the SQL-literal escaping fix), which deliberately defers the timestamp-format question (audit §6.1-2 / §7.3-13, see docs/Audit_Remediation_Results_2026-06-12.md). #146 itself changes no serialization and is backward-compatible; this note is the separate, larger "what if we ever change the time representation" picture.

Where datenum lives today

There are several distinct uses of datenum — they are independent and have very different blast radii. The two that actually represent timebases stored in data are #2 (DID SQL column) and #4–#6 (NDI/NDR global epoch times).

DID-matlab

1. did.ido identifiers — did.ido.unique_id builds IDs as [num2hex(serial_date_number) '_' num2hex(rand)], i.e. a datenum is embedded in every persistent document/branch ID. These are opaque primary keys, not a timebase. Their datenum derivation is internal and never read back as a time. Changing it would invalidate every stored ID for no benefit — explicitly out of scope for any timebase migration.
2. SQL bookkeeping timestamp column (docs / branches / branch_docs, declared timestamp NUMERIC) — written in src/did/+did/+implementations/sqlitedb.m (do_add_branch, add-doc, add-to-branch) as MATLAB now = datenum-days. DID-python writes time.time() = Unix epoch-seconds into the same column, so cross-client comparisons already diverge. Note: in DID-matlab this column is currently write-only — nothing SELECTs, sorts, or compares it — so within MATLAB the practical blast radius is low today.
3. DID document timestamp-typed fields (schema type timestamp, validated in src/did/+did/database.m via java.time.LocalDateTime.parse) — already ISO-8601 text, cross-language safe. No change needed; this is the model to follow.

NDI-matlab

4. Global epoch timebases (exp_global_time / utc) are persisted as datenum-days in the t0_t1 of each epoch_clock entry inside epoch/element documents.
   • Consumer: src/ndi/+ndi/+fun/+docTable/epoch.m reads the global-clock index with datetime(..., 'convertFrom','datenum').
   • Producers also include the import converters, e.g. +ndi/+setup/+conv/+haley/doImport.m (convertTo(datetime(...)+seconds(t0_t1_local),'datenum')) and +ndi/+setup/+conv/+babu/import.m.
   • (Device-local t0_t1, e.g. the Intan reader, is just seconds-from-zero — not affected.)
5. ndi.cache entry timestamp uses now (src/ndi/+ndi/cache.m) — in-memory only, not persisted; low concern.

NDR-matlab

6. The ABF reader is the root producer of the persisted global datenum timebase. In +ndr/+reader/axon_abf.m, get_t0_t1_from_header builds the global clock as t0t1{2} = [datenum(dt) datenum(dt+seconds(t1))] from the file's uFileStartDate, and epochclock advertises exp_global_time only when uFileStartDate is present. ABF is special because the file carries an absolute wall-clock start; most other readers only return device-local seconds.

Flow: ABF uFileStartDate → axon_abf emits exp_global_time t0_t1 as datenum → bridged via +ndi/+daq/+reader/+mfdaq/ndr.m → persisted in epoch documents' epoch_clock/t0_t1 → read back by ndi.fun.docTable.epoch with convertFrom datenum.

What a migration would require (if ever undertaken)

This is a cross-repo, cross-client change — doing it on one side alone would break the others and any already-stored data.

1. Pick a target representation, jointly for DID-matlab, DID-python, NDI-matlab, NDR-matlab, and the cloud backend. Candidates: Unix epoch-seconds (double or int64), int64 nanoseconds, or ISO-8601 TEXT (as DID already uses for timestamp-typed fields).
2. Version the format so old vs new data is distinguishable — a class_version/schema bump for documents and a format flag/migration marker for the SQL column — rather than silently reinterpreting existing numbers.
3. One-time migration of existing stored rows (SQL timestamp column) and existing documents (NDI global t0_t1).
4. Update producers and consumers in lockstep:
   • DID SQL writers in sqlitedb.m (+ DID-python equivalents) for the timestamp column.
   • NDR axon_abf.get_t0_t1_from_header / epochclock.
   • NDI import converters (+setup/+conv/...) and the ndi.fun.docTable.epoch reader, plus any analysis code that consumes global t0_t1.
5. Leave did.ido IDs alone (item #1) — out of scope.

Precision note (already understood)

datenum-days stored as a double gives only ~10 µs resolution near year 2026 (the integer day count consumes most of the mantissa) — within, but close to, the 0.1 ms spec for utc/exp_global_time in ndi.time.clocktype. Epoch-seconds (double) would give sub-µs; int64 ns would be exact. This is a motivation to keep in mind, not an urgent problem.

Summary

• DID-matlab #146 is backward-compatible and does not depend on any of this.
• The current datenum behavior is intentional; this note exists only so a future change can be made with full knowledge of the touch-points across DID, NDI, and NDR.
• No action required.

This is an informational design note with no implementation requested. For any future work, start by reading src/did/+did/+implementations/sqlitedb.m, +ndr/+reader/axon_abf.m, and src/ndi/+ndi/+fun/+docTable/epoch.m, then trace the listed import converters. Done would require an agreed cross-repository representation, versioning, migration, and coordinated producer/consumer updates.