maxbbraun / maxbbraun/llama4micro
Try larger models 💪
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 561
- Forks
- 37
- PR merge metrics
- No merged PRs in 30d
Description
The current implementation works with the 15M parameter version of tinyllamas. Just dropping in the next larger one (42M) flashes fine, but freezes at runtime.
Would need to look into what's happening here. It could be that the model weights plus the run state are larger than the available RAM (63.5MB). I might also have overlooked something about the memory layout. If it's the former, there might be a way to optimize memory usage to fit everything.
Another option would be to train a model between 15M and 42M parameters that just barely fits without any further optimizations.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the runtime freeze with the 42M-parameter tinyllamas model and comparing it with the working 15M version under the 63.5MB RAM limit. Inspect the linked memory-layout commit and determine whether the larger model can run, or whether an intermediate model is needed; done means documenting the cause and a viable model or memory approach.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- ai, embedded-iot
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100