huggingface / huggingface/ratchet

Shared Array Buffer 🪄

Open
#79 0 comments 1 reaction 0 assignees View on GitHub
help wanted
Dominant language
Rust
Stars
771
Forks
44
Avg merge
40m
Merged PRs (30d)
1

Description

When we pull the model from the wire, we should write it into a shared array buffer:
https://rustwasm.github.io/wasm-bindgen/examples/raytrace.html

This will reduce memory usage, ideally peak memory usage should be ~model size. Loading will require some magic.

Contributor guide

Open the contributing guide

Research direction

Start with the linked wasm-bindgen raytrace example, then trace how ratchet pulls the model from the wire. Done means the model is written into a shared array buffer and peak memory usage is approximately the model size; the issue does not name a file or test.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, wasm
Domain
machine-learning, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.