The RPC service contains all thread state

Problem statement

The frontend currently contains much of the state necessary to render a conversation back to the user. This is problematic in that the frontend state cannot be cleanly externalized, persisted, or restored. Aside from the management of conversation history, we have existing cases where this breaks down, such as when moving Navie into a new window in Visual Studio Code or re-arranging tabs in JetBrains IDEs. In these cases, because the frontend is reloaded, the state is lost and cannot be restored.

Proposed solution

All state should be moved into the external RPC service, in a form which can be easily (de)serialized, and supplied to the frontend at any time upon request.

My suggestion is that conversations are stored as an event log. This would mean that the frontend listens to an eventstream, provided by the RPC server, which emits any relevant state changes. It's then up to the frontend to respond to these events and render state changes accordingly.

This solution has many benefits:

• Token emission occurs through the same mechanism that performs thread re-hydration.
• Chat history can be implemented in a trivial manner via persistence and full replay of the event log.
• Distinct event types introduces a means of separation between completion tokens and user-facing feedback of state changes.

Q/A

Q: Why use an event log? Why not maintain a single state?

Let's first break this down into the two distinct problems that need to be solved:

• The state is externalized from the frontend
• The frontend is notified and responds to external state changes

The event log solves both of these with a relatively simple solution. If we opt to maintain a single golden state object, odds are that we're still emitting small delta events to the frontend as state changes occur instead of rebuilding the frontend state from a full snapshot. The latter is doubtful, because token updates are frequent enough that it's likely to hinder performance without delta compression.

Secondly, the event log allows arbitrary state changes to propagate to the frontend without the developer making changes to the thread state internal to the RPC service. For example, if a new status is reported within Navie, say "re-generating diagram", that status does not need to be integrated into any existing data structure to be persisted or propagated to the frontend.

RPC Methods

v1.navie.thread.create

Description

An alias of v1.navie.register. This RPC method is also responsible for enrolling an in-memory Thread object for listeners to subscribe to and perform actions on.

v1.navie.thread.subscribe

Description

Connect to an event stream for a given (existing) thread id. Upon connection, historical events will be emitted immediately, then the stream will continue to emit events as they occur.

Optionally, a nonce can be provided to restart the stream from some point in time. This could be useful when reconnecting to a stream, though not necessarily required, as the frontend could also perform a full replay/restore from a clean state.

*This RPC method is non-standard and does not fully comply with JSONRPC 2.0 due to the fact that it leaves the HTTP connection open as an event stream.

Parameters

• threadId the existing thread to subscribe to
• nonce (optional) identifies the point at which replay should begin

Response

Event stream

v1.navie.thread.sendMessage

Parameters

• threadId the existing thread to send a message to
• content the message to send

Response

A response is sent as soon as the user message is acknowledged or an error occurs. The response schema is one of the following types:

// The message was successfully sent
interface SuccessMessage {
  ok: true;
};

// The message failed to send, and the reason is contained within `error`
interface ErrorMessage {
  ok: false;
  error: unknown;
}

Description

Sends a user message to the given thread. The response will stream out via token events through the v1.threads.subscribe method.

v1.navie.thread.pinItem

Parameters

• threadId the existing thread to pin an item to
• operation (accepted values: pin, unpin) indicates whether or not the item is being pinned or unpinned from the thread.
• pinnedItem an object containing one of the following properties to indicate what the operation is being performed on:
  • path the path to the file
  • handle the frontend identifier of the pinned item

Response

A response is sent as soon as the user message is acknowledged or an error occurs. The response schema is one of the following types:

// Operation performed successfully
interface SuccessMessage {
  ok: true;
};

// The operation failed, and the reason is contained within `error`
interface ErrorMessage {
  ok: false;
  error: unknown;
}

Description

Pins or unpins an item within the thread.