microsoft / microsoft/TypeChat

Suggestion prompt to return UnknownText

Open
#140 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
8.7k
Forks
414
Avg merge
3h 8m
Merged PRs (30d)
3

Description

Hi, first I would like to say that this library is fantastic.

I am working on a chatbot and I want to extract the users order, below is the sample prompt I am using.

When the user says the "correct" order or something "completely off" I get a good result, but sometimes I get incorrect results

Here is the prompt:

Given the following passage:

"do you have pizza "

Respond in JSON that satisfies the following type.

// Pizza order
export type PizzaOrder = (PizzaItem | UnknownText)[];

// Use this type for order items that match nothing else
export interface UnknownText {
    type: "unknown",
    text: string; // The text that wasn't understood
}

// Pizza item with quantity, toppings and observations
export type PizzaItem = {
    quantity: number;
    toppings: PizzaTopping[];
    observation?: string; 
};

// Pizza topping  
export type PizzaTopping = "Pepperoni" | "Mushrooms" | "Black Olives" | "unknown";

If I say "I would like two pepperoni pizzas" I get the correct result

[
    {"quantity": 2,
     "toppings": ["Pepperoni"]
     }
]

And when I say "is the sky blue?" I get the correct result too "unknown"

[{
"type": "unknown",
"text": "is the sky blue"
}]

But if I say something like "Do you have pizza" I get this wrong

[{
"quantity": 1,
"toppings": ["Pepperoni"]
}]

How could I improve this prompt to get the "unknown" in this case? I am using PALM2 text-bison-001, but this also happens in Gpt-3.5-turbo-instruct too.

Thanks

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the prompt and JSON examples in the issue; no repository file or test is named. Reproduce the behavior with the stated models and determine whether TypeChat offers documented guidance or a limitation for this case. Done means a clear, reproducible recommendation or limitation is recorded.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.