HumanSignal / HumanSignal/label-studio

Preserve created_at / updated_at on annotations imported via storage sync

Open
#9,797 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
28.3k
Forks
3.7k
Avg merge
14h
Merged PRs (30d)
15

Description

## Summary

When annotations are imported into a Label Studio project via a storage sync (S3, GCS, Azure, local file — anything that hits `ImportStorage.add_task`), the `created_at` and `updated_at` values on the resulting `Annotation` rows are always set to the moment of import, overwriting any timestamps present in the source JSON.

For deployments importing historical or migrated annotation data, this silently destroys the real annotation time — the audit trail, time-based analytics, and any downstream reporting keyed on when the annotation was actually created.

I'd like to propose an **opt-in** env var — `LABEL_STUDIO_PRESERVE_IMPORT_TIMESTAMPS`, default `false` — that changes this behavior only when explicitly requested. I'm happy to open the PR.

## Current behavior

`Annotation` is defined with:

```python
created_at = models.DateTimeField(_('created at'), auto_now_add=True, ...)
updated_at = models.DateTimeField(_('updated at'), auto_now=True, ...)
```

`ImportStorage.add_task` (in `label_studio/io_storages/base_models.py`) passes each annotation dict through `AnnotationSerializer(...).save()`. The serializer treats those two fields as read-only because of the `auto_now`/`auto_now_add` flags, so any `created_at`/`updated_at` present in the source dict is silently dropped, and Django's model layer sets both to `now()` on save.

Result: importing an annotation whose source JSON says `"created_at": "2024-06-01T12:00:00Z"` produces a row with `created_at = `.

## Reproduction

1. Enable an S3 (or local) source storage on a project.
2. Sync a task whose JSON looks like:
```json
{
"data": { "image": "s3://bucket/img.png" },
"annotations": [{
"result": [...],
"created_at": "2024-06-01T12:00:00Z",
"updated_at": "2024-06-01T12:05:00Z",
"completed_by": 3
}]
}
```
3. Trigger the sync.
4. Query the resulting annotation: `created_at` and `updated_at` are the sync time, not the values from the JSON.

## Proposal

Add an opt-in Django setting sourced from an env var:

- **Name:** `LABEL_STUDIO_PRESERVE_IMPORT_TIMESTAMPS`
- **Default:** `false` — behavior identical to today.
- **When `true`:** on the annotation-creation branch of `ImportStorage.add_task`, use a subclass of `AnnotationSerializer` that makes `created_at`/`updated_at` writable, and wrap the save in a `suppress_autotime` context manager that temporarily disables the model's `auto_now`/`auto_now_add` flags. If the source JSON doesn't include one of the fields, fall back to `now()` so we never write a NULL timestamp.

Note: a `suppress_autotime` helper already exists in the codebase at `label_studio/core/old_ls_migration.py`, used by the legacy sqlite→postgres migration path. The PR would **extract that helper into a proper shared utility module** (e.g., `core/utils/db.py`) before reusing it here — new code shouldn't depend on a module named `old_ls_migration`, and this extract is a clean two-file refactor. The utility itself doesn't change.

`add_task` is the correct target for this change: it's the shared final step for the current `_scan_and_create_links` (v1) scanner and, per the design comments in the codebase, is intended to remain the endpoint for the planned `_scan_and_create_links_v2` pipeline once that's implemented. Patching there catches both flows.

Key properties of the proposal:

- **Behind a flag.** Existing users see no behavior change. Zero risk to current deployments.
- **Preserves the current serializer flow.** No bypass of `AnnotationSerializer` — `AnnotationDuplicateError` handling, FSM state init, and feature-flag validation gates all still fire.
- **Small diff.** Two commits — one small refactor to move `suppress_autotime` into a proper utils module, one focused feature commit. Together ~40 lines net plus tests.

## Why not just require users to write the timestamp via API after import?

`Annotation.updated_at` is `auto_now=True`, which fires on *every* save. Writing the timestamp after the fact requires the same `suppress_autotime` trick — you can't just PATCH it. The fix belongs at the point of import, not as a post-hoc correction.

## Impact if merged

- Deployments migrating from other annotation platforms (Prodigy, doccano, an older LS instance, in-house tooling) can preserve real annotation timestamps on import.
- Deployments backing up and restoring via JSON export/import round-trip preserve the timeline.
- The default is unchanged, so no user is surprised.

## Willingness to implement

Yes — I'll open a PR against `develop` with the change plus a test case. Just wanted an issue on file to confirm the direction is welcome before submitting.

## Environment (for context, not necessarily reproduction-critical)

- Label Studio Community edition
- Currently patched locally against a snapshot from `develop` at `1.16.0.dev0` (commit `123a32f28`, Jan 2025), preparing to rebase on nightly.
- Postgres 13 backing store.

Contributor guide

Open the contributing guide

Research direction

Start with ImportStorage.add_task in label_studio/io_storages/base_models.py and inspect suppress_autotime in label_studio/core/old_ls_migration.py. Trace the annotation-creation path and the existing migration helper before deciding how the shared utility and opt-in setting fit. Done means the flag preserves supplied timestamps without changing default behavior, while missing timestamps remain non-null; add coverage for these cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
django, python
Domain
backend, databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
63/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.