Flagsmith / Flagsmith/flagsmith-common

Refactor fetching tasks to process

Open
#75 1 comment 0 reactions 1 assignee Claimed by @emyller View on GitHub
tech-debt
Dominant language
Python
Stars
2
Forks
4
Avg merge
2d 9h
Merged PRs (30d)
10

Description

A couple issues have been observed around the [database function](https://github.com/Flagsmith/flagsmith-common/blob/9e224f121ffbb32d332f648d31b9089cacfce6c7/src/task_processor/migrations/sql/0011_get_tasks_to_process.sql) responsible for retrieving and locking tasks to process.

### 1. Result inconsistency.

Especially when running tests, somewhat often the database query will return no results, leading to flaky test failures, e.g.

```
def test_run_task_runs_task_and_creates_task_run_object_when_success(
database: str,
dummy_task: TaskHandler[[str, str]],
) -> None:
# Given
task = Task.create(dummy_task.task_identifier, scheduled_for=timezone.now())
task.save(using=database)

# When
task_runs = run_tasks(database)

# Then
> assert cache.get(DEFAULT_CACHE_KEY)
E AssertionError: assert None
E + where None = get('foo')
E + where get = .get

tests/unit/task_processor/test_unit_task_processor_processor.py:93: AssertionError
```

We became suspicious of a time zone issue so we forced `UTC` in both test databases as an experiment to confirm the bug. While the change led to tests passing consistently, eventually the same exception has randomly arose again.

We're still not sure what causes this. Time might be involved.

### 2. Locked tasks

Since the database function will select N tuples and lock them in-query, and if the associated Django queryset is not exhausted — e.g. because of an exception, we'll result in a state with tasks locked but never processed.

Related code:

https://github.com/Flagsmith/flagsmith-common/blob/ce8106418772767dcfd682bf8b95fd1d3c7c3bdc/src/task_processor/processor.py#L36-L48

The current implementation wraps running tasks within a try/except block which should prevent dangling tasks. Still, the design could be improved, and perhaps future-proofed, by using the Django ORM to somehow achieve a similar effect and efficiency.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.