apache / apache/couchdb

Evaluate the new Erlang 23+ socket module

Open
#4,739 0 comments 0 reactions 0 assignees View on GitHub
enhancement needs-triage
Dominant language
Erlang
Stars
7k
Forks
1.1k
Avg merge
1d 16h
Merged PRs (30d)
9

Description

There is a new NIF-based socket backend starting with [Erlang 23](https://www.erlang.org/blog/otp-23-highlights/#gen_tcp-and-the-new-socket-module). It might provide some benefits such as faster / lower latency connections and allow us to inspect connection state easier as there are more options/flags we can pass to recv/send calls.

It's easy enough to experiment with it by toggling: `-kernel inet_backend socket` in `vm.args` so it can be experimented with an already existing package/release.

There are however some differences as the implementation are not [100% compatible](https://www.erlang.org/doc/man/gen_tcp.html).. It involves corner cases of how closed connects are detected, how send may block vs not-block, etc.

The idea is that it might be worth at some point switching to it as the default. Before switching to it as the default we'd have to ensure the CI passes, benchmark it, and ideally a track record someone trying it out long enough on a large enough workload sample.

An example full CI run on a variety of os/platforms we support. It seems to pass the CI test well:

Screen Shot 2023-08-24 at 11 36 10 AM

(.deb packages currently fail for an unrelated reason)

Contributor guide

Open the contributing guide

Research direction

Start by enabling `-kernel inet_backend socket` in `vm.args` and running the existing CI across the supported operating systems and platforms. Compare the socket backend with the current implementation around connection closure and blocking `send` behavior, then benchmark it. Done requires passing CI, benchmark results, and evidence from a sufficiently long, large workload before considering a default switch.

Written by the indexing model from the issue text.

Assessment

Tech stack
erlang
Domain
networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.