llvm / llvm/llvm-project

mlgo AOT: serialize weights separately

Open
#199,044 0 comments 1 reaction 0 assignees View on GitHub
mlgo
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

The weights are currently exported as literals in the EmitC-generated code. If we, instead, exported them to a separate file, we could:
- have much, much smaller generated C++ files to inspect. This is valuable if we want to reason about the performance of the generated code (i.e. compiler performance)
- allow loading weights as a command line flag - as long as the model architecture is the same, weights could be dynamically loaded, as opposed to requiring a rebuild of the compiler. This can offer a training-time alternative, as well as a light-weight alternative to multi-model support. This can further enable shipping a "sufficiently good" model architecture out of the box in llvm, which could then be trained with black box techniques like Evolutionary Strategy, which have no dependency on any specific ML framework. Power users would still want to have the full flexibility of model choice and training infrastructure choice, but this would make using and training / tuning ML models accessible to more.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.