AI-assisted natural language control for GitLab/GitHub integrations (LLM Intent Layer inside Hookshot)

Nobody has claimed this yet.

## **Summary**

This proposal suggests adding an optional **AI-assisted natural-language interface** to Matrix-Hookshot so users can create and manage GitLab/GitHub issues, merge requests, and tasks using *plain human language* instead of learning bot-specific command syntax.

The key idea is to introduce a **lightweight LLM “intent parser” inside Hookshot**, which converts natural language into structured actions *while still using each user’s existing PAT-based permissions*.
This ensures correct authorship, secure execution, and seamless UX.

---

## **Problem**

Hookshot’s GitLab/GitHub integrations are powerful but depend entirely on **structured bot commands**, such as:

```
!gitlab issue new "Fix database replication"
!gitlab issue close 42
!gitlab mrs list
```

For many users (especially non-technical collaborators), remembering syntax and flags is a barrier.

More importantly:

* a **Matrix user may want to request actions naturally**, e.g.:
*“Create an urgent issue to review the MinIO backup this week.”*

* but AI bots in Matrix cannot directly trigger Hookshot as the *actual user*
(they cannot impersonate or securely reuse the user's PAT)

* if the AI bot creates the issue via the GitLab API, Hookshot cannot attribute the issue to the correct Matrix user

* therefore, today no solution provides:

* **natural language input**
* **correct GitLab authorship**
* **Hookshot-level permission enforcement**
* **secure PAT isolation**

There is a *missing architectural piece*:
Hookshot needs a way to interpret natural language **internally**, while keeping its permission model intact.

---

## **Motivation**

This feature solves several real-world issues:

1. **Lower barrier for users unfamiliar with command syntax**
Non-engineering users could participate in Issue/MR workflows more comfortably.

2. **Better UX in mobile clients**
Typing long bot commands on mobile is painful; natural language is ideal here.

3. **Safer than trying to delegate GitLab API calls to external bots**
External AI bots would need user PATs (dangerous).
Hookshot already has safe PAT storage per-MXID.

4. **Keeps authorship accurate**
Using each user’s own PAT ensures correct attribution in GitLab/GitHub.

5. **Provides future room for context-aware automation**
Example: “summarize today’s activity”, “list my overdue tasks”, “find issues mentioning Redis”.

---

## **Proposed solution: LLM Intent Layer inside Hookshot**

### **High-level idea**

Introduce an **optional AI module** in Hookshot that:

1. Receives a natural-language message from a Matrix user
2. Sends the text to an LLM (OpenAI, Azure, Anthropic, or local LLMs)
3. The LLM returns a structured “intent” in JSON
4. Hookshot validates permissions
5. Hookshot executes the action **using the user’s existing PAT**
6. Hookshot posts the result back to the room

This means *Hookshot remains the executor*, keeping all security and authorship correct.

---

## **Intent JSON format (example)**

```json
{
"action": "create_issue",
"service": "gitlab",
"project": "myorg/backend",
"title": "Review MinIO backup configuration",
"description": "",
"labels": ["infra", "urgent"]
}
```

Other intended actions could be:

```json
{
"action": "close_issue",
"issue_number": 42
}

{
"action": "list_issues",
"filters": { "state": "open", "label": "infra" }
}

{
"action": "create_mr",
"source_branch": "feature/x",
"target_branch": "main",
"title": "Add Redis metrics exporter"
}
```

---

## **Config example (new `ai:` section in config.yaml)**

```yaml
ai:
enabled: true
provider: openai
apiKey: "env:OPENAI_API_KEY"
model: "gpt-4.1-mini"
maxTokens: 512
allowedActions:
- create_issue
- close_issue
- list_issues
- create_mr
languages: ["en", "pt", "es"]
```

If `ai.enabled=false`, Hookshot behaves exactly as today (no change).

---

## **Security considerations**

1. **PATs never leave Hookshot**
The LLM never receives credentials or sensitive metadata — only the user’s natural-language request.

2. **Authorization remains 100% PAT-based**
Hookshot still checks the MXID → PAT mapping to enforce permissions.

3. **LLM can operate in “intent-only” mode**
The model is instructed to output **only JSON** with a predefined schema.
No freeform execution allowed.

4. **Admins can restrict which projects or actions the AI may trigger**
Example: allow issue creation but disallow MR merges.

5. **Support for local LLMs**
AI provider could be:

* local Ollama
* local OpenWebUI
* self-hosted LLM endpoint
avoiding external APIs entirely.

---

## **UX examples**

### **User to Hookshot:**

> “Please create a high-priority issue to review Redis replication on db26.”

### **Hookshot (LLM parsing → GitLab execution):**

```
✔ Issue #331 created: “Review Redis replication on db26”
Labels: infra, urgent
```

---

### **User:**

> “List my open tasks labeled ‘security’ for this project.”

### **Hookshot:**

```
📌 Open security issues:
#18 – Rotate SSH keys
#41 – Review firewall rules
```

---

### **User:**

> “Close the issue about MinIO space alerts.”

### **Hookshot:**

```
✔ Issue #77 closed: “Low disk space alerts on MinIO”
```

---

## **Impact**

This turns Hookshot from a command-based integration tool into a **fully conversational DevOps assistant**, while maintaining:

* existing security model
* existing action handlers
* existing PAT-based authorization
* existing configuration structure

No breaking changes to current users.

---

## **Backward compatibility**

* AI module is disabled by default
* No changes to command syntax
* No impact to current GitLab/GitHub/Jira integrations

---

## **Incremental implementation path**

1. Add `ai:` config section and basic wiring
2. Implement “intent parser” interface
3. Implement a single action: `create_issue`
4. Add more actions: `close_issue`, `list_issues`, `create_mr`
5. Add project context injection to LLM
6. Add error handling + fallbacks
7. Add support for local LLMs (Ollama/OpenWebUI endpoints)

---

## **Conclusion**

A native AI-assisted natural-language interface in Hookshot would dramatically improve developer and team productivity, especially in Matrix-centric environments.

* Zero extra bots
* Correct GitLab/GitHub authorship
* Secure PAT isolation
* Natural, intuitive user interactions
* Optional and backwards compatible

This addition aligns with Hookshot’s mission:
**connect Matrix rooms with external services in the most powerful, ergonomic way possible.**

Begin by reviewing config.yaml, the existing GitLab/GitHub action handlers, and the PAT-based authorization flow. Define the smallest increment around the proposed intent parser and create_issue action, then verify that AI is disabled by default, structured intents are authorized before execution, and existing commands remain unchanged.

Written by the indexing model from the issue text.