huggingface / huggingface/candle
how's performance on CPU (or GPU) today on ARM? (mac)
- Dominant language
- Rust
- Stars
- 21k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
Hi,
Trying to evaluate current state of performance.
Currently experimenting with starchat ggml, on CPU, M1 Mac, with pretty bad perf.
Trying to figure out how to think about performance with this library before I start porting code.
Is there a reason to expect better performance? Maybe GPU support with starchat (not supported with ggml)
Contributor guide
No contributing guide indexed for this repository
Research direction
No file, test, or entry point is named. Start by reproducing the reported starchat ggml workload on an M1 Mac and gathering the current CPU and GPU behavior; done would require a concrete, reproducible performance task or a documented answer to the support question.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- macos, rust
- Domain
- machine-learning, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100