github / github/rust-gems

bpe-openai: allow lazy loading BPE and feature flag each model

Offen
#82 3 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Rust
Sterne
134
Forks
24
Ø Merge
16 Std. 27 Min.
Gemergte PRs (30 T.)
11

Beschreibung

The fully generated `.dict` models take 70mb, which bloats binaries substantially. This is compared to 3.4mb for the targz.

It would be nice to support lazily doing this translation, which takes a modest hit to runtime performance in return for dramatically smaller binary, and the ability to only load certain models reducing memory consumption.

It would also be nice to have feature flags to turn off certain models; this is less important if we add lazy loading, though, as it would just allow you to save 1-2mb off the binary.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.