Feature: act as Make job server when executing locally

If you are building a lot of rust crates locally, they are all going to spin up at least a bunch of LLVM threads, and in future (or with nightly) they will also spin up a bunch of threads for the rustc frontend. This is a bit of an issue because buck2 will saturate its own job queue's frontier (e.g. 8 concurrent builds with -j8) and then each of those rustc invocations will try to use as many threads as it can. You can often saturate N cores by passing -j with N/2 or so. Which means in the usual case of -jN, you are really scheduling about N^2 threads at once, which is surely not great.

There is an existing protocol for limiting the parallelism of individual build actions: the GNU Make jobserver: https://www.gnu.org/software/make/manual/html_node/Job-Slots.html. Rustc and I presume a number of other build tools are able to use the jobserver if you provide the environment variable (e.g. MAKEFLAGS=--jobserver-auth=...), and do the pipe IO / windows IPC serving. This is what tools like cargo do, and of course make itself. Ultimately all the participating processes share a single semaphore.

See here for where rustc acquires and releases jobserver tokens.

The jobserver crate has a type Client that also acts as a server. If you call Client::new when starting a build, pass in the buck2 CLI's -j flag, and use its methods to attach the environment variables to local execution processes, you are basically done.

https://docs.rs/jobserver/latest/jobserver/struct.Client.html#method.new

I am sure Meta will not care about this, given you scale your builds horizontally as parallel as you like, but it would be pretty helpful for OSS builds.

Bắt đầu với tài liệu về Client::new của jobserver crate và phần được liên kết trong rustc_interface/src/util.rs để hiểu cách lấy token. Theo dõi đường dẫn thực thi cục bộ của Buck2 và cách xử lý -j của nó, sau đó gắn môi trường jobserver vào các tiến trình cục bộ. Hoàn thành khi rustc cục bộ và các công cụ tham gia khác cùng chia sẻ giới hạn song song đã cấu hình.