Reduce/manage tool definitions for faster inference and CI

How to use GitHub

• Please use the 👍 reaction to show that you are interested into the same feature.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

Feature request

Which Nextcloud Version are you currently using: v34.0.0

Is your feature request related to a problem? Please describe.

Inference and CI fails when the task processing tasks run for more than 30 minutes, which is a lot but for slower systems like the CI runners it can be exceeded, especially when the system prompt is too large.

Describe the solution you'd like

It would be nice if we could look into ways of reducing tool definitions or managing them so the system prompt size is reduced.
One way could be to group the tools so not all the definitions are dumped at once but explored by the agent in categories like Talk related tools, Calendar related tools, etc.
Although, the long run times may also be related to the token generation, shorter output would be faster if we instruct the model in that manner, and not just prompt processing of the tools.

Describe alternatives you've considered

Bigger and faster machine :)
Disabling tool calls in settings.

Additional context

https://github.com/nextcloud/context_agent/pull/196/

The issue names no files, tests, or entry points. Start by tracing how tool definitions are assembled into the system prompt and how inference and CI task timeouts are handled; done would require an agreed approach that reduces prompt or generation time without disabling tool calls.