Bogdanp / Bogdanp/django_dramatiq

How to use two queues with different configurations for processes and threads ?

Open
#75 0 comments 5 reactions 0 assignees View on GitHub
help wanted
Dominant language
Python
Stars
384
Forks
86
PR merge metrics
No merged PRs in 30d

Description

I need to run one queue exclusively with one thread and one process to enforce linearization. I run a docker container with:

```bash
./manage.py rundramatiq --processes 1 --threads 1 -Q default
```

However I need a second queue for notifications that is not restricted so I run a second container with:

```bash
./manage.py rundramatiq --processes 1 --threads 4 -Q notifications
```

The tasks work but this kind of setup is creating a collision on the task `id` at the Database level.

```
postgres_1 | 2020-08-13 21:19:44.423 UTC [686] ERROR: duplicate key value violates unique constraint "django_dramatiq_task_pkey"
postgres_1 | 2020-08-13 21:19:44.423 UTC [686] DETAIL: Key (id)=(2ac20d90-dac0-40ec-9192-e52468c21faf) already exists.
postgres_1 | 2020-08-13 21:19:44.423 UTC [686] STATEMENT: INSERT INTO "django_dramatiq_task" ("id", "status", "created_at", "updated_at", "message_data", "actor_name", "queue_name") VALUES ('2ac20d90-dac0-40ec-9192-e52468c21faf'::uuid, 'enqueued', '2020-08-13T21:19:44.371334+00:00'::timestamptz, '2020-08-13T21:19:44.373001+00:00'::timestamptz, '..... '::bytea, message_user', 'notifications')
```

Any ideas on how to setup this for my requirement and why is creating a collision when generating the ID ?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.