OpenRailAssociation / OpenRailAssociation/osrd
[meta] Improve editoast railjson performance
@Tristramg is already working on this.
Since Jul 24, 2023.
- Dominant language
- TypeScript
- Stars
- 660
- Forks
- 87
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 269
Description
https://github.com/orgs/osrd-project/projects/12/views/7?pane=issue&itemId=25095490
On a big railjson, the import lasts 3:03 minutes.
1:17 is spent on importing
0:41 is spent on generating
There seems to be a low hanging fruit on: https://github.com/osrd-project/osrd/blob/dev/editoast/src/models/infra.rs#L115-L138 where all the requests are made sequentially. There is a single postgres process at 100% during the import.
If we want to parallelize the import, we need async sql (using something else than diesel?) and we might need to split the transaction (and writing rollback code by hand — but a delete cascade should be quite trivial).
Should we accept to have two SQL connectors (diesel and sqlx ?) ? Or try to switch the whole codebase to https://github.com/weiznich/diesel_async (is would require to also change the connection pool)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.