ml-explore / ml-explore/mlx-examples
Mlx generate text, by default halusonates more
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 9k
- Forks
- 1.2k
- PR merge metrics
- No merged PRs in 30d
Description
Issue Description
Problem: When using the mix generate text command with verbose set to false, and the following parameters:
Temperature: 0.1 or 0
Top p: 1
The LLM models seem to hallucinate more frequently.
Models Used:
Mistral 7B v3
IBM Granite
Mistral 7B v2
Use Case: Text-to-SQL
Method Used: mlx.generate_text
Steps to Reproduce
Set up the environment with the specified models.
Run the mix generate text command with verbose set to false.
Use a temperature of 0.1 or 0 and top p of 1.
Observe the output for hallucinations.
Expected Behavior
The models should generate accurate and coherent SQL queries without hallucinations.
Actual Behavior
The models produce outputs that are factually incorrect or nonsensical, indicating hallucinations.
Additional Information
Dataset: 20,000 examples split for training and validation.
Hardware: 128GB M3 Max
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the issue with mlx.generate_text using verbose=false, temperature 0 or 0.1, top_p 1, and the listed Mistral or Granite models. Compare the generated Text-to-SQL output against the reported behavior and determine whether the difference is caused by the command, generation settings, or model setup. Done requires a reproducible cause and a defined correction or documented limitation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100