ml-explore / ml-explore/mlx-examples

mlx_lm with llama-3.3-70b-instruct works like base model in some case.

Open
#1,162 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
9k
Forks
1.2k
PR merge metrics
No merged PRs in 30d

Description

My prompt looks like this:

Provide a summary as well as a detail analysis of the following:
Then content to summarize goes next.

However, if I run the following,

mlx_lm.generate --model mlx-community/Llama-3.3-70B-Instruct-4bit --max-kv-size 30000 --max-tokens 2000 --temp 0.0 --top-p 0.9 --seed 1000 --system 'You are a helpful assistant' --prompt -<./28000.txt

I only get this:

"I hope this information has been helpful. If you have any further questions or need more information, please don't hesitate to ask."

I'm attaching the full prompt below.

28000.txt

Thanks!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the mlx_lm.generate command from the issue with the attached 28000.txt prompt, using the specified Llama-3.3-70B-Instruct-4bit model and options. Compare the result with the expected summary and detailed analysis, then identify whether the behavior is caused by the prompt, model handling, or generation options. Done means the cause is established and the issue has a reproducible resolution.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.