rust-lang / rust-lang/rust-analyzer

New proc-macro server RPC API

Open
#19,205 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

A-proc-macro C-Architecture
Dominant language
Rust
Stars
16.9k
Forks
2.2k
Avg merge
1d 12h
Merged PRs (30d)
72

Description

The current proc-macro server RPC API is reaching its limits preventing us from implementing more of the proc-macro API.

The current API is effectively a serial JSON based client to server request-response via stdout. That is the client (rust-analyzer, RustRover) makes a request to the proc-macro server process via JSON message and then receives a response from the server. Notably these requests cannot interleave (the proc-macro server is sequential).

This has a couple of downsides:

  • JSON: We have a weird serialization scheme for JSON to bring down the overall data we sent over the wire to combat JSON being fairly bulky. We want to replace this part (presumably with postcard)
  • client to server request-response scheme: A fairly simple messaging style that unfortunately does not suffice for implementing all the proc-macro APIs. Some of the proc-macro functions actually need to access semantic state which lives in the client and hence the server needs to be able to ask the client for this information while it is computing the response for a client's request. That means, we need a request-response scheme from client to server that allows to have server to client request-response pairs while the client is waiting for the response. One can think of this as a client request opening a communication channel from the server to the client, and the server response closing it.
  • sequential server: The current implementation of the server is entirely sequential, it can only serve a single request at any given time. Ideally we would allow the server to serve multiple requests concurrently (that is multiplex the connection), but that alone does not parallelize that well due to proc-macros being able to observe its environment and therefore we can only parallelize proc-macro expansions that share the same environment (which depending on the workload is still a win). An alternative would be to allow for spawning multiple proc-macro servers to interface with (this would more require work in the client than the server).

Notably an implementation of this needs to live side by side with the old protocol for a transition period as the proc-macro server is a rustup component that IDEs rely on.

Relevant crates:

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read the proc-macro-api crate definition and the message-handling entry point in proc-macro-srv-cli, then compare them with the existing proc-macro-srv implementation. The work is complete when a new RPC protocol supports the required communication patterns while the old protocol continues to work during the transition.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design, devtools
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.