HumanSignal / HumanSignal/label-studio
Synchronization of Local Storage Results in Database Error
- Dominant language
- TypeScript
- Stars
- 28.3k
- Forks
- 3.7k
- Avg merge
- 14h
- Merged PRs (30d)
- 15
Description
**Describe the bug**
During the synchronization process, an error consistently occurs after approximately 34 minutes, indicating that the database is locked and cannot be accessed. This prevents the successful completion of the synchronization.
**To Reproduce**
Steps to reproduce the behavior:
1. The synchronization process is initiated using a Curl.
2. After approximately 34 minutes of runtime, the error occurs stating that the database is locked.
4. This error causes the synchronization to halt and prevents it from completing.
Curl Response
```json
{
"id": 2,
"type": "localfiles",
"synchronizable": true,
"path": "/mnt/files_ki/predict",
"regex_filter": ".*(jpe?g|png)",
"use_blob_urls": true,
"last_sync": "2025-03-05T13:18:25.399310Z",
"last_sync_count": 0,
"last_sync_job": null,
"status": "completed",
"traceback": "Traceback (most recent call last):
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/backends/utils.py\", line 105, in _execute
return self.cursor.execute(sql, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/backends/sqlite3/base.py\", line 354, in execute
return super().execute(query, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sqlite3.OperationalError: database is locked
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File \"/label-studio/label_studio/io_storages/base_models.py\", line 461, in sync
import_sync_background(self.__class__, self.id)
File \"/label-studio/label_studio/io_storages/base_models.py\", line 490, in import_sync_background
storage.scan_and_create_links()
File \"/label-studio/label_studio/io_storages/localfiles/models.py\", line 100, in scan_and_create_links
return self._scan_and_create_links(LocalFilesImportStorageLink)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/label_studio/io_storages/base_models.py\", line 401, in _scan_and_create_links
task = self.add_task(data, self.project, maximum_annotations, max_inner_id, self, key, link_class)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/label_studio/io_storages/base_models.py\", line 325, in add_task
task = Task.objects.create(
^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/models/manager.py\", line 87, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/models/query.py\", line 679, in create
obj.save(force_insert=True, using=self.db)
File \"/label-studio/label_studio/tasks/models.py\", line 531, in save
super().save(*args, update_fields=update_fields, **kwargs)
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/models/base.py\", line 892, in save
self.save_base(
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/models/base.py\", line 998, in save_base
updated = self._save_table(
^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/models/base.py\", line 1161, in _save_table
results = self._do_insert(
^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/models/base.py\", line 1202, in _do_insert
return manager._insert(
^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/models/manager.py\", line 87, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/models/query.py\", line 1847, in _insert
return query.get_compiler(using=using).execute_sql(returning_fields)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/models/sql/compiler.py\", line 1836, in execute_sql
cursor.execute(sql, params)
File \"/label-studio/.venv/lib/python3.12/site-packages/sentry_sdk/utils.py\", line 1796, in runner
return sentry_patched_function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/sentry_sdk/integrations/django/__init100 5711 100 5711 0 0 2 0 0:47:35 0:34:53 0:12:42 1396^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/backends/utils.py\", line 79, in execute
return self._execute_with_wrappers(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/backends/utils.py\", line 92, in _execute_with_wrappers
return executor(sql, params, many, context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/backends/utils.py\", line 100, in _execute
with self.db.wrap_database_errors:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/utils.py\", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/backends/utils.py\", line 105, in _execute
return self.cursor.execute(sql, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/backends/sqlite3/base.py\", line 354, in execute
return super().execute(query, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.OperationalError: database is locked",
"meta": {
"attempts": 15,
"time_queued": "2025-03-05 12:43:32.588594+00:00",
"time_in_progress": "2025-03-05 12:43:32.592662+00:00",
"time_last_ping": "2025-03-05 13:18:25.114872+00:00",
"duration": 2092.806653,
"tasks_existed": 150324,
"time_completed": "2025-03-05 13:18:25.399315+00:00"
},
"title": "Input",
"description": null,
"created_at": "2025-02-19T09:07:57.571395Z",
"project": 1
}
```
Trace
```text
Traceback (most recent call last):
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/backends/utils.py\", line 105, in _execute
return self.cursor.execute(sql, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/backends/sqlite3/base.py\", line 354, in execute
return super().execute(query, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sqlite3.OperationalError: database is locked
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File \"/label-studio/label_studio/io_storages/base_models.py\", line 461, in sync
import_sync_background(self.__class__, self.id)
File \"/label-studio/label_studio/io_storages/base_models.py\", line 490, in import_sync_background
storage.scan_and_create_links()
File \"/label-studio/label_studio/io_storages/localfiles/models.py\", line 100, in scan_and_create_links
return self._scan_and_create_links(LocalFilesImportStorageLink)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/label_studio/io_storages/base_models.py\", line 401, in _scan_and_create_links
task = self.add_task(data, self.project, maximum_annotations, max_inner_id, self, key, link_class)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/label_studio/io_storages/base_models.py\", line 325, in add_task
task = Task.objects.create(
^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/models/manager.py\", line 87, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/models/query.py\", line 679, in create
obj.save(force_insert=True, using=self.db)
File \"/label-studio/label_studio/tasks/models.py\", line 531, in save
super().save(*args, update_fields=update_fields, **kwargs)
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/models/base.py\", line 892, in save
self.save_base(
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/models/base.py\", line 998, in save_base
updated = self._save_table(
^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/models/base.py\", line 1161, in _save_table
results = self._do_insert(
^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/models/base.py\", line 1202, in _do_insert
return manager._insert(
^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/models/manager.py\", line 87, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/models/query.py\", line 1847, in _insert
return query.get_compiler(using=using).execute_sql(returning_fields)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/models/sql/compiler.py\", line 1836, in execute_sql
cursor.execute(sql, params)
File \"/label-studio/.venv/lib/python3.12/site-packages/sentry_sdk/utils.py\", line 1796, in runner
return sentry_patched_function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/sentry_sdk/integrations/django/__init100 5711 100 5711 0 0 2 0 0:47:35 0:34:53 0:12:42 1396^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/backends/utils.py\", line 79, in execute
return self._execute_with_wrappers(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/backends/utils.py\", line 92, in _execute_with_wrappers
return executor(sql, params, many, context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/backends/utils.py\", line 100, in _execute
with self.db.wrap_database_errors:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/utils.py\", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/backends/utils.py\", line 105, in _execute
return self.cursor.execute(sql, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/label-studio/.venv/lib/python3.12/site-packages/django/db/backends/sqlite3/base.py\", line 354, in execute
return super().execute(query, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.OperationalError: database is locked
```
**Screenshots**
If applicable, add screenshots to help explain your problem.

**Environment (please complete the following information):**
- OS: Docker Container
- Label Studio Version 1.16.0
- Default Database Configuration(SQLite)
**Additional context**
File Count in Folder: 163348
Synced Count: 150324
Storage Configuration:

nginx additional configuration:
````text
proxy_connect_timeout 3600;
proxy_send_timeout 3600;
proxy_read_timeout 3600;
send_timeout 3600;
Contributor guide
Assessment
This issue has not been assessed yet.