huggingface / huggingface/candle

Change llama_multiprocess to multithread and execute the exp of silu to report an error.

Open
#1,385 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

It seems that tensor is dropped, and others are fine.

When run this line code
[candle-examples/examples/llama_multithread/model.rs#L177](https://github.com/mokeyish/candle/blob/llam_multithread/candle-examples/examples/llama_multithread/model.rs#L177)

it throws an error as following:
```txt
Cuda(Load {
cuda: DriverError(CUDA_ERROR_ILLEGAL_ADDRESS, "an illegal memory access was encountered"),
module_name: "uexp_bf16"
})
```

The reproduce code [candle-examples/examples/llama_multithread/main.rs](https://github.com/mokeyish/candle/blob/llam_multithread/candle-examples/examples/llama_multithread/main.rs).

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with candle-examples/examples/llama_multithread/main.rs to run the reproduction, then inspect candle-examples/examples/llama_multithread/model.rs around line 177. Trace the multiprocess-to-multithread execution and the silu exp operation; done means the reproduction no longer reports CUDA_ERROR_ILLEGAL_ADDRESS at uexp_bf16.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.