nextcloud / nextcloud/assistant

Accommodate reasoning models

Open
#285 2 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
JavaScript
Stars
88
Forks
43
Avg merge
18h 49m
Merged PRs (30d)
3

Description

Describe the feature you'd like to request

Currently, if reasoning capable (local) models are used the output will contain the <think> </think> portion of the response without any formatting.

Describe the solution you'd like

The quick "fix" would be to enable a user defined reformatting option for the outputs with specific tags (e.g. <think></think>) at the front end. It'd be neat if the output could be displayed such that the thinking portion of the response would be hidden by default but still available behind a "Click to show" -button. Including the thinking steps in the generation of future responses in the chat might also be beneficial for improved generations later on (especially when tools are used).

It's another question entirely if thinking models should actually be served using their own providers which would be capable of returning the response and thinking parts separately. The UI could then use these instead of the ordinary text processing providers if/when specified so by the admin. This would make sense as the provider side would know the output format of the model it is serving. But then again this might be too much work for a relatively small feature at this point.

Describe alternatives you've considered

The backend (LocalAI, Ollama etc.) could be used to strip the think tokens from the response altogether as it's done by the OpenAI api but then the thinking steps are "lost" for later generations and it's also interesting to view the model's internal reasoning steps every now and then.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files, tests, or entry points. Start by tracing how provider responses are rendered in the frontend and how local-model output reaches the chat UI. Done would require an agreed scope for handling sections, including whether they are hidden by default and retained for later generations.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
ai, frontend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.