cockroachdb / cockroachdb/cockroach

perf investigation: unaligned load/store on arm64

Open
#109,110 3 comments 0 reactions 0 assignees View on GitHub
A-testeng-perf C-enhancement T-testeng
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

Unaligned load/store on arm64 can lead to lower memory bandwidth and higher latency; e.g., see Go's memmove benchmarks before and after [1]. Correctness is _likely_ not an issue unless performing 64-bit _atomic_ operations on values which Go's compiler doesn't guarantee to be 64-bit aligned [2], [3]. The reason is rather subtle; it deserves a more detailed explanation.

### Correctness Explanation

Atomic load/store on arm64 must be _aligned_ (otherwise, `SIGBUS` is raised; see the writeup below). E.g., according to [4], the Store-Release instruction, `ldaddal` will fault on unaligned access,

> For Load-Acquire, Load-AcquirePC, and Store-Release instructions, the address of the data object that is supplied must be aligned to the size of the data element that is being accessed. Otherwise, the access generates an Alignment fault.

Non-atomic load/store _can_ be unaligned. Go's compiler guarantees _basic_ alignment on struct fields and array elements [5]. E.g., if a struct has a field of type `int64`, then _all_ the fields in the struct are 64-bit aligned (via padding) on arm64. However, they are _not_ 64-bit aligned on arm32, hence the "bug" note in [3],

> The first word in an allocated struct, array, or slice; in a global variable; or in a local variable (because the subject of all atomic operations will escape to the heap) can be relied upon to be 64-bit aligned.

In summary, 64-bit atomics on arm32 are _without_ faults iff they follow the above guidelines, e.g., only reference the first field of a struct. Otherwise, you risk getting shot down by an unaligned load/store. On arm64, a fault could only happen due to uses of `unsafe`, since type-checked accesses are 64-bit aligned.

[1] https://github.com/golang/go/issues/40324
[2] https://github.com/golang/go/issues/23345
[3] https://pkg.go.dev/sync/atomic#pkg-note-BUG
[4] https://developer.arm.com/documentation/102336/0100/Load-Acquire-and-Store-Release-instructions
[5] https://go.dev/ref/spec#Size_and_alignment_guarantees

Jira issue: CRDB-30784

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.