bazel-contrib / bazel-contrib/rules_python

Cannot cross-compile Gazelle extension

Open
#1,913 11 comments 3 reactions 0 assignees View on GitHub
gazelle help wanted
Dominant language
Starlark
Stars
688
Forks
721
Avg merge
14h 49m
Merged PRs (30d)
81

Description

# 🐞 bug report

### Affected Rule

The issue is caused by the rule: Gazelle extension

### Is this a regression?

Yes, before #1895, we were able to cross-compile Gazelle extension by disabling cc toolchain resolution with `--noincompatible_enable_cc_toolchain_resolution` (#1825).

### Description

The cross-compilation of Gazelle extension is broken after #1895, due to the introduction of dependency on a C library .

The issue can be partly mitigated by using [hermetic_cc_toolchain](https://github.com/uber/hermetic_cc_toolchain), which supports cross-compiling into Linux, but we still cannot cross-compile from Linux into any macOS, or from darwin_arm64 into darwin_amd64.

This is technically not a problem specific to the Gazelle extension, but a problem of the lack of C toolchain support. However, are we better off by replacing a Python dependency with a C dependency?

## 🔬 Minimal Reproduction

From the `gazelle` directory of this rules_python repo on a Linux or Apple M1 machine:

```
bazel build --platforms=@io_bazel_rules_go//go/toolchain:darwin_amd64_cgo //python:gazelle_binary
```

## 🔥 Exception or Error



ERROR: /home/user/.cache/bazel/_bazel_zplin/973b3f124c212b9dd687dfdeb379d909/external/rules_go~/BUILD.bazel:86:17: While resolving toolchains for target @@rules_go~//:cgo_context_data (d965f1d): No matching toolchains found for types @@bazel_tools//tools/cpp:toolchain_type.
To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/cpp:toolchain_type'
If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro.

## 🌍 Your Environment

**Operating System:**

* Linux
* macOS

**Output of `bazel version`:**

7.1.2

**Rules_python version:**

730a2e39bd2702910f28629d4583b3ec49f4ee5e

**Anything else relevant?**
We can register a hermetic_cc_toolchain by adding this to MODULE.bazel file:

```
bazel_dep(name = "hermetic_cc_toolchain", version = "3.1.0")

toolchains = use_extension("@hermetic_cc_toolchain//toolchain:ext.bzl", "toolchains")
use_repo(toolchains, "zig_sdk")

register_toolchains(
"@zig_sdk//toolchain:linux_amd64_gnu.2.31",
"@zig_sdk//toolchain:linux_arm64_gnu.2.31",
"@zig_sdk//toolchain:darwin_arm64",
"@zig_sdk//toolchain:darwin_amd64",
)
```

Then from a Linux machine:

```
$ bazel build --platforms=@zig_sdk//platform:darwin_arm64 //python:gazelle_binary
INFO: Analyzed target //python:gazelle_binary (133 packages loaded, 15547 targets configured).
ERROR: /home/user/go-repos/src/github.com/bazelbuild/rules_python/gazelle/python/BUILD.bazel:92:15: GoLink python/gazelle_binary_/gazelle_binary failed: (Exit 1): builder failed: error executing GoLink command (from target //python:gazelle_binary) bazel-out/k8-opt-exec-ST-13d3ddad9198/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix darwin_arm64 -arc ... (remaining 95 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
external/go_sdk/pkg/tool/linux_amd64/link: running external/hermetic_cc_toolchain~~toolchains~zig_sdk/tools/aarch64-macos-none/c++ failed: exit status 1
error: unable to find framework 'CoreFoundation'. searched paths: none
error: unable to find framework 'Security'. searched paths: none

link: error running subcommand external/go_sdk/pkg/tool/linux_amd64/link: exit status 2
Target //python:gazelle_binary failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 16.068s, Critical Path: 14.64s
INFO: 52 processes: 2 internal, 50 processwrapper-sandbox.
ERROR: Build did NOT complete successfully
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.