element-hq / element-hq/dendrite
Considerations for Go 1.19 update
- Dominant language
- Go
- Stars
- 965
- Forks
- 101
- PR merge metrics
- No merged PRs in 30d
Description
*This issue was originally created by [**@0x1a8510f2**](https://github.com/0x1a8510f2) at .*
Reading through the [Go 1.19rc1 release notes](https://tip.golang.org/doc/go1.19), I've noticed some things which may have an impact on Dendrite and may be worth considering when Go 1.19+ becomes the minimum version supported:
- The file descriptor limit increase code added in #2527 can be removed as the Go runtime will automatically increase the file descriptor limit in binaries importing the `os` package:
>On Unix operating systems, Go programs that import package [os](https://tip.golang.org/pkg/os/) now automatically increase the open file limit (RLIMIT_NOFILE) to the maximum allowed value; that is, they change the soft limit to match the hard limit. This corrects artificially low limits set on some systems for compatibility with very old C programs using the [select](https://en.wikipedia.org/wiki/Select_(Unix)) system call. Go programs are not helped by that limit, and instead even simple programs like gofmt often ran out of file descriptors on such systems when processing many files in parallel. One impact of this change is that Go programs that in turn execute very old C programs in child processes may run those programs with too high a limit. This can be corrected by setting the hard limit before invoking the Go program.
- Files with a `unix` build constraint, such as the one with the code mentioned above, can be simplified:
>The build constraint unix is now recognized in //go:build lines. The constraint is satisfied if the target operating system, also known as GOOS, is a Unix or Unix-like system. For the 1.19 release it is satisfied if GOOS is one of aix, android, darwin, dragonfly, freebsd, hurd, illumos, ios, linux, netbsd, openbsd, or solaris. In future releases the unix constraint may match additional newly supported operating systems.
- As per [arewep2pyet](https://arewep2pyet.com/), Dendrite must
>Ensure memory usage of embedded instances is bounded e.g cache sizes.
This means that the new soft memory limit may come in handy:
>The runtime now includes support for a soft memory limit. This memory limit includes the Go heap and all other memory managed by the runtime, and excludes external memory sources such as mappings of the binary itself, memory managed in other languages, and memory held by the operating system on behalf of the Go program. This limit may be managed via [runtime/debug.SetMemoryLimit](https://tip.golang.org/pkg/runtime/debug/#SetMemoryLimit) or the equivalent [GOMEMLIMIT](https://tip.golang.org/pkg/runtime/#hdr-Environment_Variables) environment variable. The limit works in conjunction with [runtime/debug.SetGCPercent](https://tip.golang.org/pkg/runtime/debug/#SetGCPercent) / [GOGC](https://tip.golang.org/pkg/runtime/#hdr-Environment_Variables), and will be respected even if GOGC=off, allowing Go programs to always make maximal use of their memory limit, improving resource efficiency in some cases. See [the GC guide](https://tip.golang.org/doc/gc-guide) for a detailed guide explaining the soft memory limit in more detail, as well as a variety of common use-cases and scenarios. Please note that small memory limits, on the order of tens of megabytes or less, are less likely to be respected due to external latency factors, such as OS scheduling. See [issue 52433](https://go.dev/issue/52433) for more details. Larger memory limits, on the order of hundreds of megabytes or more, are stable and production-ready.
Contributor guide
Research direction
Start with the Go 1.19 release notes linked in the issue, then inspect the file-descriptor limit code introduced by #2527 and Dendrite's Unix build constraints. Determine which compatibility changes and memory-limit handling are appropriate for the minimum supported Go version. Done means the required scope and implementation approach are agreed, with affected files and validation identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100