JuliaLang / JuliaLang/PackageCompiler.jl

Library architecture confusion when running on ARM M1

Open
#611 4 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Julia
Stars
1.6k
Forks
205
Avg merge
18h 37m
Merged PRs (30d)
7

Description

I am running Julia 1.6.2 (x64 mode with Rosetta 2) and having problems with `PackageCompiler.jl`.

I get warnings like `attempting to link with file built for macOS-x86_64` and when using the generated `sysimage.so`, it tries to load `sysimage.so.dylib` which is not present. I don't know why it puts `.dylib` at the end (which would be the standard library file extension on macOS).

Btw. it was working fine a few days ago and I have no idea what happened or when it eventually broke...

Here is how I build and use the sysimage (which still works nicely on the CI using Linux on x86):

```
julia --project -e 'using PackageCompiler; create_sysimage(["CSV", "Colors", "DataFrames", "Distributions", "DocOpt", "DrWatson", "HDF5", "JSON", "LaTeXStrings", "LandauDistribution", "LsqFit", "NeRCA", "PGFPlotsX", "Query", "StatsBase"]; sysimage_path="sysimage.so", precompile_statements_file="scripts/precompile.jl")'
[ Info: PackageCompiler: creating system image object file, this might take a while...
ld: warning: ignoring file /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.dylib, building for macOS-arm64 but attempting to link with file built for macOS-x86_64
ld: warning: ignoring file /Applications/Julia-1.6.app/Contents/Resources/julia/lib/libjulia.dylib, building for macOS-arm64 but attempting to link with file built for macOS-x86_64
ld: warning: ignoring file /var/folders/84/mcvklq757tq1nfrkbxvvbq8m0000gn/T/jl_iRqtyC.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
julia -J sysimage.so --project plots/cherenkov_path_length_vs_nphe.jl -o plots
ERROR: could not load library "/Users/tamasgal/Dev/PhD/sysimage.so"
dlopen(/Users/tamasgal/Dev/PhD/sysimage.so.dylib, 6): image not found
make[1]: *** [plots/cherenkov_path_length_vs_nphe.pdf] Error 1
make: *** [all] Error 2
```

I then changed the name of the system image from `sysimage.so` to `sysimage.dylib` which got rid of the file-naming issue, but then I get a `wrong architecture` message since it attempts to build for `macOS-arm64` and link with `macOS-x86_64`:

```
░ tamasgal@silentbox:PhD  master ●● 🅒 base took 7s
░ 16:36:20 > make
/Applications/Xcode.app/Contents/Developer/usr/bin/make phd_thesis.pdf
make install
julia --project -e "using Pkg; Pkg.resolve()"
No Changes to `~/Dev/PhD/Project.toml`
No Changes to `~/Dev/PhD/Manifest.toml`
julia --project -e "using Pkg; Pkg.instantiate()"
julia --project -e "using Pkg; Pkg.precompile()"
julia --project -e 'using PackageCompiler; create_sysimage(["CSV", "Colors", "DataFrames", "Distributions", "DocOpt", "DrWatson", "HDF5", "JSON", "LaTeXStrings", "LandauDistribution", "LsqFit", "NeRCA", "PGFPlotsX", "Query", "StatsBase"]; sysimage_path="sysimage.dylib", precompile_statements_file="scripts/precompile.jl")'
[ Info: PackageCompiler: creating system image object file, this might take a while...
ld: warning: ignoring file /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.dylib, building for macOS-arm64 but attempting to link with file built for macOS-x86_64
ld: warning: ignoring file /Applications/Julia-1.6.app/Contents/Resources/julia/lib/libjulia.dylib, building for macOS-arm64 but attempting to link with file built for macOS-x86_64
ld: warning: ignoring file /var/folders/84/mcvklq757tq1nfrkbxvvbq8m0000gn/T/jl_1FhnNx.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
julia -J sysimage.dylib --project plots/cherenkov_path_length_vs_nphe.jl -o plots
ERROR: could not load library "/Users/tamasgal/Dev/PhD/sysimage.dylib"
dlopen(/Users/tamasgal/Dev/PhD/sysimage.dylib, 6): no suitable image found. Did find:
/Users/tamasgal/Dev/PhD/sysimage.dylib: mach-o, but wrong architecture
/Users/tamasgal/Dev/PhD/sysimage.dylib: mach-o, but wrong architecture
make[1]: *** [plots/cherenkov_path_length_vs_nphe.pdf] Error 1
make: *** [all] Error 2
```

I am a bit puzzled why this worked before and not now, so asking for help. Maybe someone has a hint how to deal with this arch-issue `;)`

I still cannot use `Julia 1.7 for ARM (M-series)` since a few crucial packages are not available due to missing binary dependencies (e.g. `HDF5.jl`)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the reported Julia 1.6.2 command using PackageCompiler.create_sysimage and the two sysimage loading commands in the issue. Reproduce under Julia x64 through Rosetta 2 on an ARM M1 and compare the architecture warnings with the generated sysimage filename and load error. Done means the reported architecture and .so/.dylib behavior is understood and the issue has a verified resolution or documented limitation.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
build-system, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.