ml-explore / ml-explore/mlx-examples

Mlx generate text, by default halusonates more

Open
#947 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
9k
Forks
1.2k
PR merge metrics
No merged PRs in 30d

Description

Issue Description
Problem: When using the mix generate text command with verbose set to false, and the following parameters:

Temperature: 0.1 or 0
Top p: 1
The LLM models seem to hallucinate more frequently.

Models Used:

Mistral 7B v3
IBM Granite
Mistral 7B v2
Use Case: Text-to-SQL

Method Used: mlx.generate_text

Steps to Reproduce
Set up the environment with the specified models.
Run the mix generate text command with verbose set to false.
Use a temperature of 0.1 or 0 and top p of 1.
Observe the output for hallucinations.
Expected Behavior
The models should generate accurate and coherent SQL queries without hallucinations.

Actual Behavior
The models produce outputs that are factually incorrect or nonsensical, indicating hallucinations.

Additional Information
Dataset: 20,000 examples split for training and validation.
Hardware: 128GB M3 Max

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with mlx.generate_text using verbose=false, temperature 0 or 0.1, top_p 1, and the listed Mistral or Granite models. Compare the generated Text-to-SQL output against the reported behavior and determine whether the difference is caused by the command, generation settings, or model setup. Done requires a reproducible cause and a defined correction or documented limitation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.