huggingface / huggingface/candle

16GB mem unable to build candle anymore

Open
#3,540 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

HI, try build candle on my 16GB mem, it keep throw out error like this:

```
killed out of memory
```
cargo build can not succeed.

```
export NVCC_THREADS=1
1348* export MAX_JOBS=1
1349* NVCC_PREPEND_FLAGS="-t 1" cargo build --release --features "cuda flash-attn" --jobs 1
```

tried constraint threads still not work.

Why it was so strict to GPU mem when build candle.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.