bazel-contrib / bazel-contrib/rules_oci
`oci_load` is incompatible with cross architecture remote execution.
- Dominant language
- Starlark
- Stars
- 423
- Forks
- 213
- Avg merge
- 53m
- Merged PRs (30d)
- 1
Description
Hi,
Since `rules_oci` started using tar toolchain, the `oci_load` has a subtle issue with remote execution where `host_platform != exec_platform`
Consider the following example:
```
bazel run //:my_image_tar.amd64 --config=remote --extra_execution_platforms=linux_x86_64
```
The produced load script will select `exec_compatible_with` tar toolchain - `linux_x86_64` and then will be executed on host, for instance `darwin_arm64`.
I've tried to workaround this by
```
def host_exec_compatible_rule(rule, name, **kwargs):
rule(
name = name,
exec_compatible_with = [
host_cpu_constraint(),
host_os_constraint(),
],
**kwargs
)
```
However, such approach makes incorrect toolchain resolution for [building tarball itself](https://github.com/bazel-contrib/rules_oci/blob/7de244c3cdc78fb002b1b01b2806a1dc6bd5a1aa/oci/private/load.bzl#L193). The toolchains are chosen for host platform, while the rule is executed on linux_x86_64.
Similar, but not 100% same issue: https://github.com/bazelbuild/bazel/issues/22848 . The workaround with `local_only_rule` works in this case as well, but it defeats the purpose of remote execution a bit and here there is no reason to use `no-remote-exec` tag.
That said, I think the problem may be very hard to solve:
* executables (load script here) can be both:
* executed directly with `bazel run` - in this case toolchains should be selected for host
* in principle be used as dependency to a build rule - in this case, the build rule would be executed remotely, so toolchains should be selected for execution platform.
* Switching to `target_compatible_with` toolchain for the load script is probably infeasible. It would solve `bazel run` problem, but would break build rules that depend on such script.
**EDIT** I believe that if one needs to depend on the script in another rule, they can use `cfg = exec`. This makes me think that the load script should depend on tar toolchain that is `target_compatible_with`
Looking forward to your thoughts.
Contributor guide
Assessment
This issue has not been assessed yet.