OpenRailAssociation / OpenRailAssociation/osrd

[meta] Improve editoast railjson performance

Open
#4,657 1 comment 0 reactions 1 assignee View on GitHub

@Tristramg is already working on this.

Since Jul 24, 2023.

area:editoast area:railjson kind:enhancement
Dominant language
TypeScript
Stars
660
Forks
87
Avg merge
3d 4h
Merged PRs (30d)
269

Description

https://github.com/orgs/osrd-project/projects/12/views/7?pane=issue&itemId=25095490

On a big railjson, the import lasts 3:03 minutes.
1:17 is spent on importing
0:41 is spent on generating

There seems to be a low hanging fruit on: https://github.com/osrd-project/osrd/blob/dev/editoast/src/models/infra.rs#L115-L138 where all the requests are made sequentially. There is a single postgres process at 100% during the import.

If we want to parallelize the import, we need async sql (using something else than diesel?) and we might need to split the transaction (and writing rollback code by hand — but a delete cascade should be quite trivial).

Should we accept to have two SQL connectors (diesel and sqlx ?) ? Or try to switch the whole codebase to https://github.com/weiznich/diesel_async (is would require to also change the connection pool)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.