github / github/rust-gems

bpe-openai: allow lazy loading BPE and feature flag each model

Ouverte
#82 3 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Rust
Étoiles
134
Forks
24
Merge moyen
16 h 27 min
PR mergées (30 j)
11

Description

The fully generated `.dict` models take 70mb, which bloats binaries substantially. This is compared to 3.4mb for the targz.

It would be nice to support lazily doing this translation, which takes a modest hit to runtime performance in return for dramatically smaller binary, and the ability to only load certain models reducing memory consumption.

It would also be nice to have feature flags to turn off certain models; this is less important if we add lazy loading, though, as it would just allow you to save 1-2mb off the binary.

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

The issue names no files, tests, or entry points; begin by locating the bpe-openai generated .dict models and the code that translates or loads them. Done means models can be loaded lazily, individual models can be selected with feature flags, and binary and memory savings are demonstrated without losing supported behavior.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
rust
Domaine
performance
Type d'issue
Fonctionnalité
Difficulté
5/5
Temps estimé
Plus d'une semaine
Activité
À l'abandon
Clarté
À clarifier
Accessibilité débutants
30/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.