google / google/gvisor

Tune number of channels used in lisafs

Open
#6,313 2 comments 0 reactions 1 assignee Claimed by @ayushr2 View on GitHub
area: filesystem type: enhancement
Dominant language
Go
Stars
19.3k
Forks
2k
Avg merge
3d 5h
Merged PRs (30d)
264

Description

9P uses a maximum of 4 channels per gofer mount. Re-evaluate if this upper bound should be increased to support more concurrent RPCs.

The rationale for keeping it limited to 4 is that increasing this could lead to more memory usage by the sentry as described [here](https://github.com/google/gvisor/blob/c16e69a9d5ec3422b648a6d32842442925285a29/pkg/p9/transport_flipcall.go#L27-L43). Each channel mmaps a 1MB region.

However, that memory is not used allocated until it is touched. lisafs and p9 both maintain a stack of available channels. So for sequential non-concurrent RPCs, the same channel will be re-used (hence not requiring the host to allocate the other channel's pages). The mmap-ed region for all channels will only be allocated if the application ends up making 4 concurrent RPCs requiring all 4 channels to get activated.

Another point to note is that most messages are quite small in size (lesser than 1KB). Only payload based messages (like read, write, getdents) require more memory than that. Even in that for fsgofer the common case is that read/write go through the host FD avoiding the channel altogether. Even if all channels are used, only those pages touched in that 1MB range will be allocated.

So should we increase the number of channels used by each gofer mount in lisafs? Will we incur a massive memory penalty for it? Will it significantly improve performance for filesystem intensive workloads? (abseil build might be a good workload to answer these questions as bazel highly parallelizes its compilation/building work which require a lot of file operations)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.