Rust-GPU / Rust-GPU/rust-gpu

[Migrated] Optimize `spirv-builder` clean build wall time

Offen
#112 5 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Dieses Issue hat noch niemand übernommen.

Vorherrschende Sprache
Rust
Sterne
3.4k
Forks
126
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

Issue automatically imported from old repo: https://github.com/EmbarkStudios/rust-gpu/issues/858

Originally creatd by repi on 2022-04-10T11:01:16Z


We should look into improve our clean build times of when building spriv-builder & rustc_codegen_spirv as that is the main scenario in CI and also useful for users if our crates rebuild fast.

Right now it looks like we have a couple of quite unfortunate serial dependencies that significantly increases the wall time. Here is the current results with cargo build --release -p spirv-builder -Z timings on my AMD 5950x (32 vCPU):

image

Potential optimizations

  • Could spirv-builder be the one depending on spirv-tools instead of rustc_codegen_spirv?
    • I.e. move the responsibility of running spirv-opt/spirv-val and final linking to the builder.
    • Does that sound like it could be doable @eddyb ?
  • Remove/replace serde, quite crazy it takes 20 seconds to compile
    • Think we only have two use cases, our ModuleResult output and then the internal decorations that is using more complex serde implementation (CustomDecoration).
    • have no idea how serde+serde_json is taking such a crazy long time to build here though, can't see any other crates enabling tons of types that serde derive macros are used on or similar
    • ModuleResult would be trivial to replace serde with nanoserde, could work for CustomDecoration also?
    • #860
  • Remove/replace sanitize-filename dependency that compiles and uses regex crate for super trivial things
  • Optimize builds of rspirv itself somehow?
    • Unfortunate that it is so slow to build by itself but could be hard to fix and tons of types in it. No open issues on it in the rspirv repo as far as I could see
  • Determine why rustc_codegen_spirv is not split in frontend/codegen sections in the profile report.
    • This would enable spriv-builder itself to start building earlier and before the codegen of rustc_codegen_spirv is done.
    • Update: This is not possible as it it needs to be built as a dylib

If we do get rid of serde and manage to make sure rustc_codegen_spirv doesn't have to wait for the spirv-tools-sys build, we could get get a 15+ second wall time improvement here as rustc_codegen_spirv could start as soon as frontend section of rspirv has been built:
- image

Beitragsleitfaden

Beitragsleitfaden öffnen

Erste Schritte

  1. Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
  3. Forke das Repository und arbeite in einem Branch.
  4. Öffne einen Pull Request, der die Issue-Nummer nennt.

Rechercherichtung

Führe zunächst cargo build --release -p spirv-builder -Z timings aus und untersuche die Abhängigkeitszeiten eines sauberen Builds für spirv-builder und rustc_codegen_spirv. Lies die Abhängigkeitsbeziehungen rund um spirv-tools, serde, serde_json, sanitize-filename, regex und rspirv; abgeschlossen ist die Aufgabe, wenn eine messbare Verringerung der verstrichenen Zeit eines sauberen Builds mit einer klar ausgewählten Optimierung erreicht wurde.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
rust
Bereich
build-system, performance
Issue-Typ
Refactoring
Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Aktivitätsstatus
Veraltet
Klarheit
Muss geklärt werden
Anfängerfreundlichkeit
25/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.