huggingface / huggingface/transformers.js

[Question] Chat models with `System messages` for templated responses?

Open
#348 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
16.3k
Forks
1.2k
Avg merge
6d 2h
Merged PRs (30d)
6

Description

## A super interesting story of mine 👇

As I have recently started learning to use models from hugging face I also tried directing models to understand templates of their expected responses. So far I only created a prompt which included instructions concatenated with user's input, which had the following shape:

Prompt = you are a model who plays a role of a mother. Your child refers to you with the following phrase: {input}. Respond to it while being loving, caring and affectionate.

This would work, but as learned after, it is not how things are done. To take more control over a model there are so called system messages, which are, as I understood, part of conversational/chat models ONLY. I still do not quite understand the difference between these two though.

As I understood, System messages, are a specific feature, they are also consumed by the model in the first place, much like a ruler's will or the god's word. And unlike system messages, simply expanding a prompt with additional clarification like "please do blablabla using this data: {input}" is absolutely not how responses are meant to be templated, right?

I would rather send a model both a system message and a user's prompt every time. First, the model reads a system message defining the response template in such a way:
```
System message: You are a helpful JSON generator who thinks over a description of an object and returns a JSON. Always return a JSON only without any additional clarifications and supportive phrases. Your task is not to engage in the lively dialogue but rather always return JSON objects.
```
With a system message attached, I also give it a user's prompt as it is:
```
Prompt: A house with 2 doors and 3 windows and a garage with a car.
```

## My current situation

Now you may ask, why do I bother about system messages and directing a model? I answer, I do that because I need a model to return an object generated from user's description (eg. a house with 3 doors and 2 windows) as a pure JSON string ONLY. Yes, I am creating a test app which thinks over a description to render it as a JSON object. Why do I need accuracy? Because the model's response is not going to appear in the chat box, there is none actually, instead it is going to be consumed by the function, and I want to maximise the chance up to 100% that a string response is a valid JSON object convertible to a JavaScript object with the use of `JSON.parse(res)`.

## The question is

How do I normally use a hugging face chat model and feed it system + user messages?

Also, did I get everything right?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.