aws / aws/aws-lc

FIPS build fails with `zig cc`

Open
#3,261 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Assembly
Stars
830
Forks
212
Avg merge
2d 22h
Merged PRs (30d)
61

Description

### Problem:

An invalid `libbcm_c_generated_asm.a` is generated when `bcm.c` is compiled with Zig, leading to `delocate` failing with a parse error. This only happens when `-DFIPS=1` is set.

```
error while parsing "/work/build/linux-arm64-fips/cmake-build/crypto/fipsmodule/libbcm_c_generated_asm.a":
parse error near Unknown (line 1 symbol 1 - line 1 symbol 1):
""
```

I'm working with `83d85826595641bb2f23c6108f2e031fcd5cd505` commit, as that's what `aws-lc-fips-sys` `0.13.14` mentions in [`Cargo.toml`](https://docs.rs/crate/aws-lc-fips-sys/0.13.14/source/Cargo.toml).

I'm sharing a minimal reproduction of the issue: https://github.com/RafalSumislawski/aws-lc-zig-fips-repro

#### Why it fails?

AWS-LC's FIPS CMakeList defines a normal static library target named [`bcm_c_generated_asm`](https://github.com/aws/aws-lc/blob/83d85826595641bb2f23c6108f2e031fcd5cd505/crypto/fipsmodule/CMakeLists.txt#L338-L344) from [`crypto/fipsmodule/bcm.c`](https://github.com/aws/aws-lc/blob/83d85826595641bb2f23c6108f2e031fcd5cd505/crypto/fipsmodule/bcm.c), adds the regular include/definition setup for that target, and sets [`COMPILE_OPTIONS "-S"`](https://github.com/aws/aws-lc/blob/83d85826595641bb2f23c6108f2e031fcd5cd505/crypto/fipsmodule/CMakeLists.txt#L345-L352) so the compiler emits assembly instead of an object file. As a result the `bcm.c` compile command includes both assembly-output and object/dependency flags:

```sh
zig-cc ... -S -MD -MT ... -MF ... -o bcm.c.o -c bcm.c
```

GCC/Clang tolerate this command shape and emit assembly. Zig does not produce an archive member that `delocate` can parse, so the archive starts with object-like data instead of assembly text.

The build then runs [`delocate`](https://github.com/aws/aws-lc/blob/83d85826595641bb2f23c6108f2e031fcd5cd505/crypto/fipsmodule/CMakeLists.txt#L367). That command passes the generated static archive with [`-a $`](https://github.com/aws/aws-lc/blob/83d85826595641bb2f23c6108f2e031fcd5cd505/crypto/fipsmodule/CMakeLists.txt#L368-L375), so `delocate` expects the archive member produced by `bcm_c_generated_asm` to be textual assembly.

#### Fix?

This can either be seen as a `zig cc` issue: "`zig cc` doesn't behave exactly like clang"
Or an AWS-LC issue: "A compiler command containing `-S` as well as `-MD`/`-MT`/`-MF` is questionable, and just happens to work with clang"

I managed to build aws-lc with zig using shims that removed the surplus flags.

So this could be fixed here in aws-lc, by generating `bcm.c` textual assembly with a dedicated custom command that does not include `-c`, `-MD`, `-MT`, or `-MF`, rather than relying on the current normal CMake object/static-library compile rule plus [`COMPILE_OPTIONS "-S"`](https://github.com/aws/aws-lc/blob/83d85826595641bb2f23c6108f2e031fcd5cd505/crypto/fipsmodule/CMakeLists.txt#L351).

#### Context (or why do I care about using zig here)

I'm trying to build FIPS-compliant rust AWS lambda functions using `cargo lambda`. Using zig as C compiler is the default https://www.cargo-lambda.info/guide/cross-compiling.html#cross-compiling-with-the-zig-toolchain and in my experience most reliable way of cross compiling rust lambdas across arm64/x86_64 MacOS/Linux

#### Relevant details
AWS-LC commit: 83d85826595641bb2f23c6108f2e031fcd5cd505

System information: I've provided a reproduction in a docker container. My local setup is an M1 Mac, but it's mostly irrelevant

Build log:
```text
-- Go compiler 1.22.2 found
-- The C compiler identification is Clang 21.1.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/local/bin/zig-cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- FIPS build mode configured
-- FIPS entropy source method configured: Passive
-- The CXX compiler identification is Clang 21.1.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/local/bin/zig-c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Perl: /usr/bin/perl (found version "5.38.2")
-- Setting CMAKE_C_STANDARD=11
-- stdalign_check.c probe is positive, enabling AWS_LC_STDALIGN_AVAILABLE
-- builtin_swap_check.c probe is positive, enabling AWS_LC_BUILTIN_SWAP_SUPPORTED
-- linux_u32.c probe is positive, enabling AWS_LC_URANDOM_U32
-- The ASM compiler identification is Clang with GNU-like command-line
-- Found assembler: /usr/local/bin/zig-cc
CMake Deprecation Warning at cmake/go.cmake:55 (cmake_policy):
The OLD behavior for policy CMP0116 will be removed from a future version
of CMake.

The cmake-policies(7) manual explains that the OLD behaviors of all
policies are deprecated and that a policy should be set to OLD only under
specific short-term circumstances. Projects should be ported to the NEW
behavior and not rely on setting a policy to OLD.
Call Stack (most recent call first):
crypto/fipsmodule/CMakeLists.txt:367 (go_executable)

CMake Deprecation Warning at cmake/go.cmake:55 (cmake_policy):
The OLD behavior for policy CMP0116 will be removed from a future version
of CMake.

The cmake-policies(7) manual explains that the OLD behaviors of all
policies are deprecated and that a policy should be set to OLD only under
specific short-term circumstances. Projects should be ported to the NEW
behavior and not rely on setting a policy to OLD.
Call Stack (most recent call first):
crypto/fipsmodule/CMakeLists.txt:415 (go_executable)

-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Configuring done (21.6s)
-- Generating done (0.1s)
-- Build files have been written to: /work/build/linux-arm64-fips/cmake-build
Change Dir: '/work/build/linux-arm64-fips/cmake-build'

Run Build Command(s): /usr/bin/ninja -v crypto
[1/325] cd /work/build/linux-arm64-fips/cmake-build/crypto && /usr/bin/cmake -E make_directory test && /usr/bin/perl /work/aws-lc/crypto/test/asm/trampoline-armv8.pl linux64 test/trampoline-armv8.S

...

[317/325] cd /work/build/linux-arm64-fips/cmake-build/crypto/fipsmodule && /usr/bin/cmake -E make_directory . && /usr/bin/perl /work/aws-lc/crypto/fipsmodule/aes/asm/vpaes-armv8.pl linux64 vpaes-armv8.S
[318/325] cd /work/build/linux-arm64-fips/cmake-build/crypto/fipsmodule && /usr/bin/cmake -E make_directory . && /usr/bin/perl /work/aws-lc/crypto/fipsmodule/sha/asm/sha512-armv8.pl linux64 sha512-armv8.S
[319/325] cd /work/aws-lc/crypto/fipsmodule && /usr/bin/go build -o /work/build/linux-arm64-fips/cmake-build/crypto/fipsmodule/delocate boringssl.googlesource.com/boringssl/util/fipstools/delocate && /usr/bin/go run /work/aws-lc/util/godeps.go -format depfile -target crypto/fipsmodule/delocate -pkg boringssl.googlesource.com/boringssl/util/fipstools/delocate -out /work/build/linux-arm64-fips/cmake-build/crypto/fipsmodule/delocate.d
[320/325] cd /work/build/linux-arm64-fips/cmake-build/crypto/fipsmodule && ./delocate -a /work/build/linux-arm64-fips/cmake-build/crypto/fipsmodule/libbcm_c_generated_asm.a -o bcm-delocated.S -cc /usr/local/bin/zig-cc -cc-flags \ \ -Wno-newline-eof\ -Wa,--noexecstack /work/aws-lc/include/openssl/arm_arch.h /work/aws-lc/include/openssl/asm_base.h /work/aws-lc/include/openssl/target.h aesv8-armx.S aesv8-gcm-armv8.S aesv8-gcm-armv8-unroll8.S armv8-mont.S bn-armv8.S ghash-neon-armv8.S ghashv8-armx.S keccak1600-armv8.S md5-armv8.S p256-armv8-asm.S p256_beeu-armv8-asm.S sha1-armv8.S sha256-armv8.S sha512-armv8.S vpaes-armv8.S p384/bignum_add_p384.S.S p384/bignum_sub_p384.S.S p384/bignum_neg_p384.S.S p384/bignum_tomont_p384.S.S p384/bignum_deamont_p384.S.S p384/bignum_montmul_p384.S.S p384/bignum_montmul_p384_alt.S.S p384/bignum_montsqr_p384.S.S p384/bignum_montsqr_p384_alt.S.S p384/bignum_nonzero_6.S.S p384/bignum_littleendian_6.S.S p521/bignum_add_p521.S.S p521/bignum_sub_p521.S.S p521/bignum_neg_p521.S.S p521/bignum_mul_p521.S.S p521/bignum_mul_p521_alt.S.S p521/bignum_sqr_p521.S.S p521/bignum_sqr_p521_alt.S.S p521/bignum_tolebytes_p521.S.S p521/bignum_fromlebytes_p521.S.S curve25519/bignum_mod_n25519.S.S curve25519/bignum_neg_p25519.S.S curve25519/bignum_madd_n25519.S.S curve25519/bignum_madd_n25519_alt.S.S curve25519/edwards25519_decode.S.S curve25519/edwards25519_decode_alt.S.S curve25519/edwards25519_encode.S.S curve25519/edwards25519_scalarmulbase.S.S curve25519/edwards25519_scalarmulbase_alt.S.S curve25519/edwards25519_scalarmuldouble.S.S curve25519/edwards25519_scalarmuldouble_alt.S.S curve25519/curve25519_x25519_byte.S.S curve25519/curve25519_x25519_byte_alt.S.S curve25519/curve25519_x25519base_byte.S.S curve25519/curve25519_x25519base_byte_alt.S.S fastmul/bignum_kmul_16_32.S.S fastmul/bignum_kmul_32_64.S.S fastmul/bignum_ksqr_16_32.S.S fastmul/bignum_ksqr_32_64.S.S fastmul/bignum_emontredc_8n.S.S generic/bignum_ge.S.S generic/bignum_mul.S.S generic/bignum_optsub.S.S generic/bignum_sqr.S.S fastmul/bignum_kmul_16_32_neon.S.S fastmul/bignum_kmul_32_64_neon.S.S fastmul/bignum_ksqr_16_32_neon.S.S fastmul/bignum_ksqr_32_64_neon.S.S fastmul/bignum_emontredc_8n_neon.S.S generic/bignum_copy_row_from_table.S.S generic/bignum_copy_row_from_table_8n_neon.S.S generic/bignum_copy_row_from_table_16_neon.S.S generic/bignum_copy_row_from_table_32_neon.S.S
FAILED: crypto/fipsmodule/bcm-delocated.S /work/build/linux-arm64-fips/cmake-build/crypto/fipsmodule/bcm-delocated.S
cd /work/build/linux-arm64-fips/cmake-build/crypto/fipsmodule && ./delocate -a /work/build/linux-arm64-fips/cmake-build/crypto/fipsmodule/libbcm_c_generated_asm.a -o bcm-delocated.S -cc /usr/local/bin/zig-cc -cc-flags \ \ -Wno-newline-eof\ -Wa,--noexecstack /work/aws-lc/include/openssl/arm_arch.h /work/aws-lc/include/openssl/asm_base.h /work/aws-lc/include/openssl/target.h aesv8-armx.S aesv8-gcm-armv8.S aesv8-gcm-armv8-unroll8.S armv8-mont.S bn-armv8.S ghash-neon-armv8.S ghashv8-armx.S keccak1600-armv8.S md5-armv8.S p256-armv8-asm.S p256_beeu-armv8-asm.S sha1-armv8.S sha256-armv8.S sha512-armv8.S vpaes-armv8.S p384/bignum_add_p384.S.S p384/bignum_sub_p384.S.S p384/bignum_neg_p384.S.S p384/bignum_tomont_p384.S.S p384/bignum_deamont_p384.S.S p384/bignum_montmul_p384.S.S p384/bignum_montmul_p384_alt.S.S p384/bignum_montsqr_p384.S.S p384/bignum_montsqr_p384_alt.S.S p384/bignum_nonzero_6.S.S p384/bignum_littleendian_6.S.S p521/bignum_add_p521.S.S p521/bignum_sub_p521.S.S p521/bignum_neg_p521.S.S p521/bignum_mul_p521.S.S p521/bignum_mul_p521_alt.S.S p521/bignum_sqr_p521.S.S p521/bignum_sqr_p521_alt.S.S p521/bignum_tolebytes_p521.S.S p521/bignum_fromlebytes_p521.S.S curve25519/bignum_mod_n25519.S.S curve25519/bignum_neg_p25519.S.S curve25519/bignum_madd_n25519.S.S curve25519/bignum_madd_n25519_alt.S.S curve25519/edwards25519_decode.S.S curve25519/edwards25519_decode_alt.S.S curve25519/edwards25519_encode.S.S curve25519/edwards25519_scalarmulbase.S.S curve25519/edwards25519_scalarmulbase_alt.S.S curve25519/edwards25519_scalarmuldouble.S.S curve25519/edwards25519_scalarmuldouble_alt.S.S curve25519/curve25519_x25519_byte.S.S curve25519/curve25519_x25519_byte_alt.S.S curve25519/curve25519_x25519base_byte.S.S curve25519/curve25519_x25519base_byte_alt.S.S fastmul/bignum_kmul_16_32.S.S fastmul/bignum_kmul_32_64.S.S fastmul/bignum_ksqr_16_32.S.S fastmul/bignum_ksqr_32_64.S.S fastmul/bignum_emontredc_8n.S.S generic/bignum_ge.S.S generic/bignum_mul.S.S generic/bignum_optsub.S.S generic/bignum_sqr.S.S fastmul/bignum_kmul_16_32_neon.S.S fastmul/bignum_kmul_32_64_neon.S.S fastmul/bignum_ksqr_16_32_neon.S.S fastmul/bignum_ksqr_32_64_neon.S.S fastmul/bignum_emontredc_8n_neon.S.S generic/bignum_copy_row_from_table.S.S generic/bignum_copy_row_from_table_8n_neon.S.S generic/bignum_copy_row_from_table_16_neon.S.S generic/bignum_copy_row_from_table_32_neon.S.S
error while parsing "/work/build/linux-arm64-fips/cmake-build/crypto/fipsmodule/libbcm_c_generated_asm.a":
parse error near Unknown (line 1 symbol 1 - line 1 symbol 1):
""

ninja: build stopped: subcommand failed.
```

Contributor guide

Open the contributing guide

Research direction

Start in crypto/fipsmodule/CMakeLists.txt at the bcm_c_generated_asm target and the delocate command, then inspect crypto/fipsmodule/bcm.c and reproduce the FIPS build with zig cc using the linked minimal reproduction. Compare the generated archive and compiler command with a working compiler. Done means delocate completes and the FIPS build succeeds without the invalid archive parse error.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, zig
Domain
build-system, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.