bazel-contrib / bazel-contrib/rules_oci

Issue running arm64 images from multi-arch builds done on linux x86_64 hosts with `platform_transition_filegroup`

Open
#372 7 comments 2 reactions 0 assignees View on GitHub
need: investigation
Dominant language
Starlark
Stars
423
Forks
213
Avg merge
53m
Merged PRs (30d)
1

Description

I'm running into an issue with multi-arch builds using `platform_transition_filegroup`. Minimal repro [here](https://github.com/macourteau/aspect-rules_oci).

That repo sets up a `js_binary`, which is packaged in a `js_image_layer` and then an `oci_image`. It then goes through `platform_transition_filegroup`s to transition the image to both `arm64` and `x86_64` architectures, and these transitioned filegroups get packaged into an `oci_image_index`, which is pushed with `oci_push`.

Details of the issue are in the README.md file in that repo, but I'm copying the relevant bits here as well:

* If I build on an **arm64 macOS host** and `bazel run //:push_app -- --tag macos_arm64`, I'm able to `docker run :macos_arm64` on both `arm64` and `x86_64` hosts.
* If I build on an **x86_64 macOS host** and `bazel run //:push_app -- --tag macos_x84_64`, I'm also able to `docker run :macos_x86_64` on both `arm64` and `x86_64` hosts.
* If I build on an **x86_64 linux host** (say from inside the `build-container`) and `bazel run //:push_app -- --tag ubuntu_x86_64`, then I'm able to `docker run :ubuntu_x86_64` on an `x86_64` host, but on an `arm64` host, I get an error message when running the container:
```
$ docker run --rm -it :ubuntu_x86_64
qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory
```

### Result on arm64 macOS host

```
$ docker run --rm :macos_arm64
Unable to find image ':macos_arm64' locally
macos_arm64: Pulling from
20274425734a: Already exists
fea475dd86f0: Pull complete
ff126526266e: Pull complete
Digest: sha256:7506ab201004e78326416ef8e43ce09403ff99cd5a48fd9f3fa69af3698a500c
Status: Downloaded newer image for :macos_arm64
______________
< Hello there! >
--------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
$ docker run --rm :macos_x86_64
Unable to find image ':macos_x86_64' locally
macos_x86_64: Pulling from
Digest: sha256:43efd29c437781693d7af917f06206888876ff7a88a22a11ffbf9369c89653e9
Status: Downloaded newer image for :macos_x86_64
______________
< Hello there! >
--------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
$ docker run --rm :ubuntu_x86_64
Unable to find image ':ubuntu_x86_64' locally
ubuntu_x86_64: Pulling from
20274425734a: Already exists
e9ee3bab9d5e: Pull complete
848ef9faff35: Pull complete
Digest: sha256:48780564428b4a464ac5c4ecb38195cb0e401f041870b364b21b8cb5256e8c83
Status: Downloaded newer image for :ubuntu_x86_64
qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory
```

### Result on x86_64 macOS host

```
$ docker run --rm :macos_arm64
Unable to find image ':macos_arm64' locally
macos_arm64: Pulling from
445a6a12be2b: Already exists
1ccca8531fae: Pull complete
ff126526266e: Pull complete
Digest: sha256:7506ab201004e78326416ef8e43ce09403ff99cd5a48fd9f3fa69af3698a500c
Status: Downloaded newer image for :macos_arm64
______________
< Hello there! >
--------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
$ docker run --rm :macos_x86_64
Unable to find image ':macos_x86_64' locally
macos_x86_64: Pulling from
Digest: sha256:43efd29c437781693d7af917f06206888876ff7a88a22a11ffbf9369c89653e9
Status: Downloaded newer image for :macos_x86_64
______________
< Hello there! >
--------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
$ docker run --rm :ubuntu_x86_64
Unable to find image ':ubuntu_x86_64' locally
ubuntu_x86_64: Pulling from
Digest: sha256:48780564428b4a464ac5c4ecb38195cb0e401f041870b364b21b8cb5256e8c83
Status: Downloaded newer image for :ubuntu_x86_64
______________
< Hello there! >
--------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
```

Contributor guide

Open the contributing guide

Research direction

Start with the linked repository's README.md and its use of js_binary, js_image_layer, oci_image, platform_transition_filegroup, oci_image_index, and oci_push. Reproduce the Bazel run from the build-container on an x86_64 Linux host, compare it with the macOS cases, and verify that the resulting ubuntu_x86_64 image runs on both x86_64 and arm64 hosts.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, linux
Domain
build-system, devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.