posit-dev / posit-dev/positron

Supervisor: Connect to remote supervisor and kernels

Open
#8,300 3 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area: kallichore area: remote host
Dominant language
TypeScript
Stars
4.3k
Forks
184
Avg merge
1d 11h
Merged PRs (30d)
206

Description

Currently, Positron always uses a local supervisor to run R and Python kernels.

However, all communication with the supervisor, the kernels, and other protocols (LSP, DAP, etc.) already happens over TCP. Consequently, we could add a way to connect to a remote supervisor that works very similarly to the local supervisor. Most code in Positron wouldn't need to change.

This would allow us to run just the kernels and supervisor remotely, which is an improvement over Remote SSH in many contexts:

  • Resource constraints: the remote SSH backend is enormous, uses Node, and has hefty memory requirements as well; it is sufficiently unwieldy that many containers and VMs running Positron have little to no space left for actual user data. Kallichore + the kernels are just a few MB, leaving plenty of memory for computations.
  • Permission constraints: users might not have SSH access to the machine, or might not have enough permissions to install server software onto it. SSH access is a hard requirement for Remote SSH.
  • Security constraints: the remote SSH backend has a very large surface area and may be deemed unacceptably risky for installation into remote servers in regulated environments.
  • Licensing constraints: the remote SSH backend is basically a full copy of Positron, which means it can't be used in all the same places as the kernels (which are MIT licensed).
Currently
graph LR
p[Positron] -- HTTP --> k[Kallichore]
p -- WebSocket --> k
k -- ZeroMQ --> r[R Session]
k -- ZeroMQ --> py[Python Session]
r -- TCP LSP --> p
py -- TCP LSP --> p
Proposed alternative
graph LR
subgraph "Local machine"
p[Positron] 
end
subgraph "Remote machine"
p -- HTTP --> k[Kallichore]
p -- WebSocket --> k
k -- ZeroMQ --> r[R Session]
k -- ZeroMQ --> py[Python Session]
r -- TCP LSP --> p
py -- TCP LSP --> p
end
Considerations
  • This would introduce some versioning considerations; right now there is very little infrastructure to test/support different versions of the supervisor (and kernels) since we bundle everything together
  • It would be useful to (optionally) mimic some of the features of Remote SSH, such as tunnelling the connections over SSH, automatically installing the supervisor and kernels, etc.
  • It is awkward to have the LSPs running on a different machine than the Positron extension host, because they will be looking at a different file system & machine environment.
  • The idea of connecting to a "remote kernel" is not new. In fact it is already supported in VS Code -- you can connect to a Jupyter Server to run a kernel there.
  • We may want to have a different UI treatment for remote kernels to help set expectations around how they will work, and to disable LSPs if they won't work against local files, etc.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are named. Start by tracing how Positron currently connects to the local Kallichore supervisor and its HTTP, WebSocket, LSP, and DAP protocols. Done would require an agreed design and implementation for remote supervisor and kernel connections, including versioning and the stated UI and filesystem considerations.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend-api-design, devtools, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.