Static builds on MacOS ARM64 (Apple Silicon aka M1/M2) don't seem to work as expected

Description of the problem or steps to reproduce

Building on MacOS ARM64 does not work with static compilation.

To Reproduce
Steps to reproduce the behavior:

1. Use MacOS ARM64
2. make

Error:

go build -trimpath -ldflags "-s -w -X github.com/zyedidia/micro/v2/internal/util.Version=2.0.12-dev.55 -X github.com/zyedidia/micro/v2/internal/util.CommitHash=651a3010 -X 'github.com/zyedidia/micro/v2/internal/util.CompileDate=April 30, 2023' -linkmode external -extldflags -Wl,-sectcreate,__TEXT,__info_plist,/tmp/micro-info.plist" ./cmd/micro
# github.com/zyedidia/micro/v2/cmd/micro
loadinternal: cannot find runtime/cgo

Hi!

I've been trying to build micro on my M1 (also called Apple Silicon, the newer models are M2.. it's really just AArch64/ARM64), and was getting super obtuse/misleading errors when using the Makefile.

This resulted in researching the fun that is ARM64 MacOS and static linking with Go. I haven't played with Go much, but I have played with llvm etc to know where to dig into why it's not working.. luckily I like deep diving into random topics, and tonights was about static linking madness with MacOS.

Here's what I get when trying to run make build-quick on Darwin ARM64:

$ go version
go version go1.20.3 darwin/arm64

make build-quick

go build -trimpath -ldflags "-s -w -X github.com/zyedidia/micro/v2/internal/util.Version=2.0.12-dev.55 -X github.com/zyedidia/micro/v2/internal/util.CommitHash=651a3010 -X 'github.com/zyedidia/micro/v2/internal/util.CompileDate=April 30, 2023' -linkmode external -extldflags -Wl,-sectcreate,__TEXT,__info_plist,/tmp/micro-info.plist" ./cmd/micro
# github.com/zyedidia/micro/v2/cmd/micro
loadinternal: cannot find runtime/cgo

Confusing error message, as it leads to so many different answers! This works fine when building on a Linux host, as the builds there aren't static:

GOOS=linux GOARCH=amd64 go generate ./runtime
go build -trimpath -ldflags "-s -w -X github.com/zyedidia/micro/v2/internal/util.Version=2.0.12-dev.55 -X github.com/zyedidia/micro/v2/internal/util.CommitHash=651a3010 -X 'github.com/zyedidia/micro/v2/internal/util.CompileDate=April 30, 2023' " ./cmd/micro

That led me down a rabbit hole, and trying to figure out what's going on. Here's a few things I understand:

1. Code signing is different with Darwin on ARM64. Linkers such as zld have had issues, and other binaries would just return an error on compilation. It's confusing when it happens. See Issue #42684. This is kind of related to this issue (especially when starting to dig), but I don't think this is the actual issue.

2. There is a fantastic blog post about static compilation with MacOS ARM64 & Go by Matt Turner, see here. Very informative read and good background information about the issue.

3. There is a fantastic blog post about static compilation in C on MacOS, which should give you more of a background: https://michaelspangler.io/posts/statically-linking-on-macos.html

tldr; Static linking on MacOS can be painful, especially when it references a library that isn't static.

So trying to compile statically ON MacOS ARM64 doesn't work, as it tries to static compile, where as other operating systems don't when cross-compiling.

So yeah, it's tricky. It seems that static compiling on ARM64 MacOS is a bit of a mess, and you can't just static link certain things. 🤷.

Fiddling with the flags in tools/info.plist gives me this error, which is closer to the actual issue..

/opt/homebrew/Cellar/go/1.20.3/libexec/pkg/tool/darwin_arm64/link: running cc failed: exit status 1
ld: library not found for -lcrt0.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Apparently this wasn't supported by gcc shipped by Apple.

This option will not work on Mac OS X unless all libraries (including libgcc.a) have also been compiled with -static. Since neither a static version of libSystem.dylib nor crt0.o are provided, this option is not useful to most people.

I kept trying both the homebrew version of go and the "official" version to see if they return different errors, however they both result in the same errors.

Development builds vs "production builds"

Usually when you are developing vs when you actually want a "production build", you're in the "Write code" -> "Compile" -> "Test change" feedback loop, building without static linking is fine, as this would increase compilation time anyway, right?

When would you want a static build?

This would be for someone who just wants to download and use micro on Darwin ARM64 and not worry about something weird breaking. This is a good thing.. if it worked :-).

Relevant Go issues for this

https://go-review.googlesource.com/c/go/+/477839

So what's the fix?

I remember reading a while ago that you can add which libraries can be static, and which can be dynamic. For MacOS, this should work.

Which frameworks/libraries do we need to add?

Great post here: https://stackoverflow.com/questions/844819/how-to-static-link-on-os-x

On my MBA M1, running Ventura 13.3, if I remove the static linking part:

otool -L micro

micro:
    /usr/lib/libresolv.9.dylib (compatibility version 1.0.0, current version 1.0.0)
    /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1971.0.0)
    /System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 60420.101.2)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.100.3)

So IMHO, playing with flags like -lresolv etc in tools/info-plist.go is the way to go, then you can define which libraries can be static and which ones are dynamic.

Start by reproducing make or make build-quick on Darwin ARM64 with the Go build shown in the issue. Inspect the Makefile and tools/info-plist.go, then compare the generated build flags with the runtime/cgo and linker errors. Done means the appropriate build target completes on Apple Silicon without these static-linking failures.