nextcloud / nextcloud/assistant

Make agent tools more discoverable in the chat with AI feature

Open
#310 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
88
Forks
43
Avg merge
18h 49m
Merged PRs (30d)
3

Description

Problem

Some of the tasks available in the Assistant tabs are already embedded in the Chat with AI feature when agency is enabled.
Users are more accustomed to use chats when working with AI, so it would be nice to make these features more discoverable there.


@marcelklehr on the things that would be already available in chat:

Any of our text2text tasks are possible in chat even without the context_agent app installed. The user may, however, not know an optimal prompt for instructing the AI and may have to do multiple attempts. Our ready-made task types (tabs in the assistant) are probably easier to use, but also less flexible.

If the context_agent app is installed, the following tools are available in chat that are also available as assistant tabs:

  • Transcribe a media file
  • Generate documents
    • Example prompt: "Can you generate me a slide deck for my presentation about cats?"
    • Example prompt: "Can you generate me a spreadsheet with some plausible numbers for countries and their population count?"
    • Example prompt: "Can you generate me a pdf with an outline about what to see in Berlin?"
  • Generate images
    • Example prompt: "Can you generate me an image of a cartoon drawing of a roman soldier typing something on a laptop?"
  • Get contents of a file (and optionally do some text processing with it)
    • Example prompt: "Can you summarize the following file in my documents? Design/Planning.md"

Complete list of available tools that the agent can use https://docs.nextcloud.com/server/latest/admin_manual/ai/app_context_agent.html

  • Todo: group all these tools into categories into action cards that can be example prompts in the chat with UI interface
    As a guide for grouping: each file in this folder is a group of tasks

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the tool-group files in ex_app/lib/all_tools and the linked Context Agent documentation to identify the available task categories and prompts. Then trace the chat UI entry point and define action cards for those groups; done means the relevant tools are grouped and discoverable as example prompts in chat.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
ai, frontend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.