open-compass / open-compass/VLMEvalKit
Could you support custom system prompt?
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.4k
- Forks
- 768
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 17
Description
Is there support for adding custom system prompt like R1?
SYSTEM_PROMPT = (
"A conversation between User and Assistant. The user asks a question, and the Assistant solves it. The assistant "
"first thinks about the reasoning process in the mind and then provides the user with the answer. The reasoning "
"process and answer are enclosed within <think> </think> and <answer> </answer> tags, respectively, i.e., "
"<think> reasoning process here </think><answer> answer here </answer> {Question}"
)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating where model prompts are constructed and passed to the evaluation flow. Check how the existing R1 integration formats its prompt, then determine where a user-supplied system prompt would need to enter. Done means custom system-prompt text is accepted and used without breaking the existing prompt behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100