hellas-ai / hellas-ai/catgrad

feat(catgrad-llm): support thinking switches and structured input/outputs

Open
#457 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
34
Forks
7
PR merge metrics
No merged PRs in 30d

Description

some models have a "thinking" mode that can be toggled on/off- qwen
some support high/med/low- gpt-oss

openai/anthropic conversation types have a 'thinking' content type- check if/where this is supported as an input to the template and render it correctly
do we need to parse model output to construct the typed 'thinking' outputs that the proprietary apis return? do other servers (e.g llamacpp) do this? if so, same logic as #454 applies- if we did not request thinking but model outputs thinking.. what do we do? render as text output?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing how templates accept OpenAI/Anthropic conversation content and how model responses are represented, then compare other servers such as llamacpp and the decisions in #454. Done means the thinking switch levels, structured thinking inputs, and typed or text outputs have defined behavior across the supported cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
ai, backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.