simonw / simonw/llm

Truncated output with Llama model

Open
#561 0 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
12.5k
Forks
998
Avg merge
6d 12h
Merged PRs (30d)
15

Description

I'm experiencing what appears to be truncation or aborted evaluation when using the Meta-Llama-3-8B-Instruct model from gpt4all:

Here's an example of what I mean:

$ llm 'Write a small JavaScript program that asks trivia questions'

 Here is a simple JavaScript program that asks trivia questions:

// Define the questions and answers
var questions = [
  { question: "What is the capital of France?", answer: "Paris" },
  { question: "Who painted the Mona Lisa?", answer: "Leonardo da Vinci" },
  { question: "What is the largest planet in our solar system?", answer: "Jupiter" }
];

// Define a function to ask a question
function askQuestion(question) {
  console.log(question.question);
  var response = prompt("Enter your answer:");
  if (response.toLowerCase() === question.answer.toLowerCase()) {
    console.log("Correct!");
  } else {
    console.log(`Sorry, that's incorrect. The correct answer is ${question.answer}.`);
  }
}

// Ask each question
questions.forEach(askQuestion);

This program defines an array of questions and answers, then uses a function `askQuestion` to ask each question and check the user's

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the reported llm command with the Meta-Llama-3-8B-Instruct model from gpt4all and compare the output with the shown truncation. Trace the command's model evaluation path to determine whether the incomplete response comes from the CLI or the model integration; done means the cause is identified and the output behavior is verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.