microsoft / microsoft/TypeChat

Better prompt for chatglm2-6b-4bit

Open
#119 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Suggestion
Dominant language
TypeScript
Stars
8.7k
Forks
414
Avg merge
3h 8m
Merged PRs (30d)
3

Description

Description:

When I run sentiment example, the response result from chatglm is always '{\nsentiment : "xxx"\n}' which parse failed by JSON.Parse().

Environment:

I used fastchat and chatglm2-6b-4bit to mock openai api

Root cause:

Because chatglm2-6b-4bit has lower performance than chatgpt, it cannot return the correct result via the example

Solution:

Add one prompt in source code "Please note that ..."

function createRequestPrompt(request) {
        return `You are a service that translates user requests into JSON objects of type "${validator.typeName}" according to the following TypeScript definitions:\n` +
            `\`\`\`\n${validator.schema}\`\`\`\n` +
            `Please note that the response string can be parsed to JSON object via JSON.Parse() such as {"aaa" : "bbb"}\n` +
            `The following is a user request:\n` +
            `"""\n${request}\n"""\n` +
            `The following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n`
    }
Result:

Now the response result from chatglm is '{"sentiment": "xxx"}' which can be parsed to JSON correctly

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Locate the createRequestPrompt(request) function shown in the issue and read the surrounding request-generation code. Update the prompt so the model returns JSON that JSON.parse() can consume, then run the sentiment example or existing validation checks to confirm the response format.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.