ml-explore / ml-explore/mlx-examples
mlx_lm with llama-3.3-70b-instruct works like base model in some case.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 9k
- Forks
- 1.2k
- PR merge metrics
- No merged PRs in 30d
Description
My prompt looks like this:
Provide a summary as well as a detail analysis of the following:
Then content to summarize goes next.
However, if I run the following,
mlx_lm.generate --model mlx-community/Llama-3.3-70B-Instruct-4bit --max-kv-size 30000 --max-tokens 2000 --temp 0.0 --top-p 0.9 --seed 1000 --system 'You are a helpful assistant' --prompt -<./28000.txt
I only get this:
"I hope this information has been helpful. If you have any further questions or need more information, please don't hesitate to ask."
I'm attaching the full prompt below.
Thanks!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the mlx_lm.generate command from the issue with the attached 28000.txt prompt, using the specified Llama-3.3-70B-Instruct-4bit model and options. Compare the result with the expected summary and detailed analysis, then identify whether the behavior is caused by the prompt, model handling, or generation options. Done means the cause is established and the issue has a reproducible resolution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100