code-corps / code-corps/code-corps-api

Fix deadlocks caused by EctoOrdered

Abierto
#1,147 0 comentarios 0 reacciones 0 asignados Ver en GitHub
needs estimate
Lenguaje dominante
Elixir
Estrellas
234
Forks
82
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

# Problem

Our new sync process revealed some issues caused in our use of `EctoOrdered` when there are a large number of writes. At the kind of scale we need for highly concurrent GitHub syncing, we are hitting deadlocks in the database.

Here's an example of the errors:

```
2017-11-02T23:39:42.632335Z [error] Task #PID<0.22638.6> started from #PID<0.3430.0> terminating
** (Postgrex.Error) ERROR 40P01 (deadlock_detected): deadlock detected

Process 14260 waits for ShareLock on transaction 1981764; blocked by process 14258.
Process 14258 waits for ShareLock on transaction 1981765; blocked by process 14260.
(ecto) lib/ecto/adapters/sql.ex:440: Ecto.Adapters.SQL.execute_or_reset/7
(ecto_ordered) lib/ecto_ordered.ex:242: EctoOrdered.increment_other_ranks/2
(ecto_ordered) lib/ecto_ordered.ex:172: EctoOrdered.ensure_unique_position/2
(ecto) lib/ecto/repo/schema.ex:456: anonymous fn/2 in Ecto.Repo.Schema.run_prepare/2
(elixir) lib/enum.ex:1811: Enum."-reduce/3-lists^foldl/2-0-"/3
(ecto) lib/ecto/repo/schema.ex:268: anonymous fn/14 in Ecto.Repo.Schema.do_update/4
(ecto) lib/ecto/repo/schema.ex:768: anonymous fn/3 in Ecto.Repo.Schema.wrap_in_transaction/6
(ecto) lib/ecto/adapters/sql.ex:576: anonymous fn/3 in Ecto.Adapters.SQL.do_transaction/3
(db_connection) lib/db_connection.ex:1275: DBConnection.transaction_run/4
(db_connection) lib/db_connection.ex:1199: DBConnection.run_begin/3
(db_connection) lib/db_connection.ex:790: DBConnection.transaction/3
(code_corps) lib/code_corps/github/sync/issue/task/task.ex:51: CodeCorps.GitHub.Sync.Issue.Task.find_or_create_task/2
(elixir) lib/enum.ex:1255: Enum."-map/2-lists^map/1-0-"/2
(elixir) lib/enum.ex:1255: Enum."-map/2-lists^map/1-0-"/2
(code_corps) lib/code_corps/github/sync/issue/task/task.ex:45: CodeCorps.GitHub.Sync.Issue.Task.sync_project_github_repo/1
(code_corps) lib/code_corps/github/sync/sync.ex:178: CodeCorps.GitHub.Sync.sync_project_github_repo/1
(elixir) lib/task/supervised.ex:85: Task.Supervised.do_apply/2
(stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Function: #Function<3.4867021/0 in CodeCorpsWeb.ProjectGithubRepoController.create/2>
Args: []
```

One approach to this is to rewrite our card syncing to use a single `position` column with floats and some ordering scheme [like this one](https://stackoverflow.com/questions/29791543/how-does-trello-handle-rearrangement-of-cards-lists-checklists-etc), with most of the operations happening on the server itself.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.