When I implemented Llama3-8B-1.58-100B-tokens with TL1 on ARM, a malloc() error was occured.
Open
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 40.3k
- Forks
- 3.7k
- PR merge metrics
- No merged PRs in 30d
Description
Environment
- CPU: NVIDA Grace CPU(72thread)
- Model: Llama3-8B-1.58-100B-tokens with TL1
- Prompt: AI is going to
- N_Predict: 128
- Threads: 1, 2, 4, 8, 16, 32, 64, 72
- Context Size: 2048
- Temperature: 0.8
When I implemented Llama3-8B-1.58-100B-tokens with TL1 on ARM, a malloc() error was occured.
- Error Type
- Type 1: malloc() error and nothing output
- Type 2: malloc() error and correct output(generated text)
- Type 3: double malloc() error and correct output(generated text)
| Thread Size | Error Message | Output | Error Type |
|---|---|---|---|
| 1thread | malloc(): invalid next size (unsorted) Error occurred while running command: Command '['build/bin/llama-cli', '-m', '/root/BitNet/models/Llama3-8B-1.58-100B-tokens/ggml-model-tl1.gguf', '-n', '128', '-t', '1', '-p', 'AI is going to', '-ngl', '0', '-c', '2048', '--temp', '0.8', '-b', '1']' died with <Signals.SIGABRT: 6>. | Nothing | Type 1 |
| 2thread | free(): invalid next size (normal) Error occurred while running command: Command '['build/bin/llama-cli', '-m', '/root/BitNet/models/Llama3-8B-1.58-100B-tokens/ggml-model-tl1.gguf', '-n', '128', '-t', '2', '-p', 'AI is going to', '-ngl', '0', '-c', '2048', '--temp', '0.8', '-b', '1']' died with <Signals.SIGABRT: 6>. | Generated Text | Type 2 |
| 4thread | double free or corruption (!prev) Error occurred while running command: Command '['build/bin/llama-cli', '-m', '/root/BitNet/models/Llama3-8B-1.58-100B-tokens/ggml-model-tl1.gguf', '-n', '128', '-t', '4', '-p', 'AI is going to', '-ngl', '0', '-c', '2048', '--temp', '0.8', '-b', '1']' died with <Signals.SIGABRT: 6>. | Generated Text | Type 3 |
| 8thread | double free or corruption (!prev) Error occurred while running command: Command '['build/bin/llama-cli', '-m', '/root/BitNet/models/Llama3-8B-1.58-100B-tokens/ggml-model-tl1.gguf', '-n', '128', '-t', '8', '-p', 'AI is going to', '-ngl', '0', '-c', '2048', '--temp', '0.8', '-b', '1']' died with <Signals.SIGABRT: 6>. | Generated Text | Type 3 |
| 16thread | double free or corruption (!prev) Error occurred while running command: Command '['build/bin/llama-cli', '-m', '/root/BitNet/models/Llama3-8B-1.58-100B-tokens/ggml-model-tl1.gguf', '-n', '128', '-t', '16', '-p', 'AI is going to', '-ngl', '0', '-c', '2048', '--temp', '0.8', '-b', '1']' died with <Signals.SIGABRT: 6>. | Generated Text | Type 3 |
| 32thread | double free or corruption (!prev) Error occurred while running command: Command '['build/bin/llama-cli', '-m', '/root/BitNet/models/Llama3-8B-1.58-100B-tokens/ggml-model-tl1.gguf', '-n', '128', '-t', '32', '-p', 'AI is going to', '-ngl', '0', '-c', '2048', '--temp', '0.8', '-b', '1']' died with <Signals.SIGABRT: 6>. | Generated Text | Type 3 |
| 64thread | free(): invalid next size (normal) Error occurred while running command: Command '['build/bin/llama-cli', '-m', '/root/BitNet/models/Llama3-8B-1.58-100B-tokens/ggml-model-tl1.gguf', '-n', '128', '-t', '64', '-p', 'AI is going to', '-ngl', '0', '-c', '2048', '--temp', '0.8', '-b', '1']' died with <Signals.SIGABRT: 6>. | Generated Text | Type 2 |
| 72thread | double free or corruption (!prev) Error occurred while running command: Command '['build/bin/llama-cli', '-m', '/root/BitNet/models/Llama3-8B-1.58-100B-tokens/ggml-model-tl1.gguf', '-n', '128', '-t', '72', '-p', 'AI is going to', '-ngl', '0', '-c', '2048', '--temp', '0.8', '-b', '1']' died with <Signals.SIGABRT: 6>. | Generated Text | Type 3 |
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the reported llama-cli command on the NVIDA Grace ARM environment with the Llama3-8B-1.58-100B-tokens TL1 model, varying the listed thread counts. The issue names no source files or tests, so first locate the inference and ARM execution entry points involved in the command. Done means all thread counts complete without malloc or free corruption errors and still produce output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100