nextcloud / nextcloud/assistant

Chat User Instructions for Chat Completions Not Being Sent to LLM with context_agent enabled

Open
#338 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug priority normal
Dominant language
JavaScript
Stars
88
Forks
43
Avg merge
18h 49m
Merged PRs (30d)
3

Description

Which version of integration_openai are you using?

3.7.1

Which version of Nextcloud are you using?

NC Hub 10 31.0.7

Which browser are you using? In case you are using the phone App, specify the Android or iOS version and device please.

Vivaldi

Describe the Bug

The Chat User Instructions for Chat Completions setting is not being passed to configured LLM. The following task printouts show there is no 'system prompt' which should be included per the User Instruction setting.

Here is the specific instruction I set for before each test request.

Image

Then I ran these test requests. I see that any query to the Chat with AI mode in NC Assistant creates at least 2 tasks. I believe this is to account for tool calling when supported. Also note that no request has any 'system prompt' included.

Request 1 Task:
- id: 10929
- type: core:contextagent:interaction
- lastUpdated: 1754668023
- status: STATUS_SUCCESSFUL
- userId: rockyreidel
- appId: assistant:chatty-llm
- input:
- input: What is your name?
- confirmation: 0
- conversation_token: {}
- output:
- output: My name is Assistant. How can I assist you today?
- conversation_token: ...

  - actions:
  - sources:
- customId: chatty-llm:50
- completionExpectedAt: 1754668075
- progress: 1
- scheduledAt: 1754668015
- startedAt
- endedAt: 1754668023
- error_message
  • 1:
    • id: 10930
    • type: core:text2text:chatwithtools
    • lastUpdated: 1754668020
    • status: STATUS_SUCCESSFUL
    • userId: rockyreidel
    • appId: context_agent
    • input:
      • system_prompt:
        You are a helpful AI assistant with access to tools, please respond to the user's query to the best of your ability, using the provided tools! If you used a tool, you still need to convey its output to the user.
        Use the same language for your answers as the user used in their message.
        Today is 2025-08-08.
        Detect the language the user is using. Reply in the detected language. Do not output the detected language.
        Only use the duckduckgo_results_json tool if the user explicitly asks for a web search.
        You can check which conversations exist using the list_talk_conversations tool, if a conversation cannot be found.
        You can check which calendars exist using the list_calendars tool, if a calendar can not be found.
        you can find out a user's email address and location by using the find_person_in_contacts tool.
        you can find out the current user's location by using the find_details_of_current_user tool.
        If an item should be added to a list, check list_calendars for a fitting calendar and add the item as a task there.

      • input: What is your name?

      • tool_message:

      • history: ...

      • - output:
        
      • output: My name is Assistant. How can I assist you today?

      • tool_calls:

    • customId:
    • completionExpectedAt: 1754668036
    • progress: 1
    • scheduledAt: 1754668018
    • startedAt: 1754668018
    • endedAt: 1754668020
    • error_message

Request Task 2:
0:
- id: 10935
- type: core:contextagent:interaction
- lastUpdated: 1754668352
- status: STATUS_SUCCESSFUL
- userId: rockyreidel
- appId: assistant:chatty-llm
- input:
- input: What is your name? What is the color of the sky?
- confirmation: 0
- conversation_token: {}
- output:
- output: My name is Assistant. The color of the sky is generally blue during the day and changes to shades of orange, pink, and red during sunrise and sunset, depending on weather and atmospheric conditions.
- conversation_token: ....
- - actions:
- sources:
- customId: chatty-llm:51
- completionExpectedAt: 1754668403
- progress: 1
- scheduledAt: 1754668343
- startedAt
- endedAt: 1754668352
- error_message

  • 1:
    • id: 10936
    • type: core:text2text:chatwithtools
    • lastUpdated: 1754668349
    • status: STATUS_SUCCESSFUL
    • userId: rockyreidel
    • appId: context_agent
    • input:
      • system_prompt:
        You are a helpful AI assistant with access to tools, please respond to the user's query to the best of your ability, using the provided tools! If you used a tool, you still need to convey its output to the user.
        Use the same language for your answers as the user used in their message.
        Today is 2025-08-08.
        Detect the language the user is using. Reply in the detected language. Do not output the detected language.
        Only use the duckduckgo_results_json tool if the user explicitly asks for a web search.
        You can check which conversations exist using the list_talk_conversations tool, if a conversation cannot be found.
        You can check which calendars exist using the list_calendars tool, if a calendar can not be found.
        you can find out a user's email address and location by using the find_person_in_contacts tool.
        you can find out the current user's location by using the find_details_of_current_user tool.
        If an item should be added to a list, check list_calendars for a fitting calendar and add the item as a task there.

      • input: What is your name? What is the color of the sky?

      • tool_message:

      • history:.....

      • - output:
        
      • output: My name is Assistant. The color of the sky is generally blue during the day and changes to shades of orange, pink, and red during sunrise and sunset, depending on weather and atmospheric conditions.

      • tool_calls:

    • customId:
    • completionExpectedAt: 1754668364
    • progress: 1
    • scheduledAt: 1754668346
    • startedAt: 1754668347
    • endedAt: 1754668349
    • error_message

Any other settings I should troubleshoot to find the reason why this setting is not being passed correctly?

Expected Behavior

The setting from Chat User Instructions for Chat Completions in the Admin Settings > Artificial Intelligence should be included in the requests to the LLM

To Reproduce

Change setting Chat User Instructions for Chat Completions
Send test request to LLM
Review task process if a system prompt was included or not

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at Admin Settings > Artificial Intelligence and trace how Chat User Instructions for Chat Completions flow into the context_agent interaction and core:text2text:chatwithtools task. Compare the configured instruction with the task's system_prompt for a reproduced request. Done means the configured instruction appears in the LLM request's system prompt.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
ai, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.