FlowFuse / FlowFuse/flowfuse

FE: Migrate Expert communication from HTTP to MQTT over WebSockets

Open
#6,897 0 comments 0 reactions 0 assignees View on GitHub
area:ff-expert
Dominant language
JavaScript
Stars
400
Forks
89
Avg merge
1d 21h
Merged PRs (30d)
146

Description

**Parent Deliverable:** FlowFuse/product#47 — Expert as Agentic Co-pilot

**Parent Initiative:** FlowFuse/product#39 — FlowFuse Expert scaffolds user ideas into deployed instances

## Context

The FlowFuse Expert's frontend currently communicates with the AI Agent backend via HTTP streaming. This architecture has reached its limits:

- **HTTP request timeouts** on long-running AI tasks (flow generation, multi-step agentic workflows)
- **No native streaming** in OpenAI format — the current workaround is fragile
- **No bidirectional communication** — HTTP is request/response; the Expert can't receive push updates from the agent
- **`custom:get-canvas-flows`** (read live canvas state) is **blocked** by this migration — it requires real-time state sync that HTTP can't support

The instance logs system already uses MQTT over WebSockets (via EMQX) for both remote and hosted instances, providing proven infrastructure we can extend.

## Architecture

Based on the March 13 and March 11 alignment sessions:

```
┌─────────────────┐ EMQX Broker ┌─────────────────────┐
│ AI Agent │◄──────────────────────►│ Front-End Expert │
│ (Instance A) │ #/x1/expert/ │ (FlowFuse app) │
│ │ / │ │
│ │ {agent|expert} │ │
└────────┬────────┘ └──────────┬───────────┘
│ │
│ 1. get credentials │ 1. get credentials
│ from Forge BE │ from Forge BE
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────────┐
│ Forge BE │ │ Front-End │
│ │ │ nr-assistant Plugin │
│ │ │ (Node-RED in iFrame)│
└─────────────────┘ │ AKA target instance │
└─────────────────────┘
┌─────────────────┐
│ MCP Server │
│ Flow Builder │
│ (Instance C) │
└─────────────────┘
```

**Topic structure:**
- `#/x1/expert//agent` — messages from Expert to Agent
- `#/x1/expert//expert` — messages from Agent to Expert
- Wildcard subscription supported for session management

**Key decisions:**
- **Session ID only** (not instance ID) — moved to chat-session-id to avoid confusion about what "instance" refers to
- **Forge BE controls credentials** — both AI Agent and Front-End Expert get MQTT credentials from Forge BE, preventing unauthorized topic subscription
- **Chat persistence** — MQTT enables session rehydration (new sessions from old session parts)

## Scope

### Core Migration

Replace HTTP with MQTT over WebSockets in the Expert's frontend communication:

1. **MQTT client integration** — check device Node-RED logs component for reference on how to pull WS over MQTT for the frontend
2. **Replace HTTP with MQTT** — replicate current capabilities (send questions, receive streamed responses)
3. **Subscribe/Emit to channels** — implement pub/sub pattern for Expert ↔ Agent communication
4. **Credential flow** — integrate with Forge BE for MQTT authentication

### Custom Action Migration

Each custom action needs to work over the MQTT transport. Sub-tasks with estimates from refinement:

- [ ] `custom:add-nodes` — Add nodes to the canvas (1h)
- [ ] `custom:remove-nodes` — Remove nodes from the canvas (1h)
- Note: ensure nodes are not created/shown on top of one another (initial tests OK, needs validation for complex flows)
- [ ] `custom:update-node` — Update node properties in place (1h)
- "These will not be 'bottleneck' — check out the definitions/docs is essential"
- [ ] `custom:set-wires` — Set outbound wires for a node (1h)
- Need to check for removal and editing of wires
- [ ] `custom:add-workspace` — Add a new tab to the canvas (1h)
- [ ] `custom:remove-workspace` — Remove a tab from the canvas (1h)
- [ ] `custom:import-flow` — Bulk import a flow (nodes + tabs + wires + subflows) (3h)
- Need to investigate subflow mechanism of import
- [ ] `custom:get-canvas-flows` — Read live canvas state (3h)
- **Currently blocked** by this migration (WebSocket migration planned)
- [ ] Anything else discovered during migration (4h buffer)

### Exploration Tasks

- [ ] MQTT pubsub exploration (2h) — investigate MQTT client libraries, connection handling, reconnection
- [ ] Forge level exploration (4h) — understand Forge BE credential provisioning, auth flow, topic ACLs

## Out of Scope

- Multi-modality support (PDFs, images, video, audio over MQTT) — future iteration, noted as consideration
- Shared sessions across users — noted for future, needs rehydration design
- VueX to Pinia migration — can be done in tandem but tracked separately
- AI Agent ↔ EMQX integration (Steve's side)
- EMQX auth configuration (Ben's side)

## Dependencies

| Dependency | Owner | Status |
|---|---|---|
| AI Agent MQTT integration with EMQX | Steve | Planned (next week per whiteboard) |
| EMQX auth / credential provisioning | Ben | Planned |
| Forge BE credential endpoint | Steve/Ben | Needed |
| Instance logs MQTT reference implementation | — | Exists (reference for patterns) |

## Security Considerations

- **Topic isolation**: Forge BE must enforce that a client can only subscribe to their own chat-session-id topics
- **Credential rotation**: MQTT credentials should be scoped and time-limited
- **Context poisoning risk**: noted in discussion — need to prevent cross-session data leakage via MQTT topic subscription
- **Faking with logs**: MQTT topic subscription could potentially be abused — Forge BE must validate session ownership

## Technical Notes

- Instance logs already make use of this MQTT-over-WebSocket setup (both remote & hosted) — use as reference implementation
- VueX/Pinia migration can be done in tandem with this work
- The `custom:get-canvas-flows` action is the most impactful unlock — enables reading live canvas state for the Expert's context awareness
- Consider context poisoning prevention: ensure chat history is scoped per session

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.