microsoft / microsoft/onnxruntime
[Build] ONNX fails to build with CUDA 13.3
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the issue
Trying to build onnxruntime 1.26.0 with CUDA 13.3.0 fails with an error message related to template instantiation in Abseil:
```
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/cuda/13.3.0-46dd8459fe712b1803d2e4e8089fe749/bin/nvcc -forward-unknown-to-host-compiler -DCOMPILE_HOPPER_TMA_GEMMS -DCPUINFO_SUPPORTED -DCPUINFO_SUPPORTED_PLATFORM=1 -DEIGEN_MPL2_ONLY -DEIGEN_USE_THREADS -DENABLE_BF16 -DENABLE_CPU_FP16_TRAINING_OPS -DENABLE_CUDA_NHWC_OPS -DENABLE_DLPACK -DENABLE_FP4 -DENABLE_FP8 -DEXCLUDE_SM_110 -DEXCLUDE_SM_86 -DHAS_SM80_OR_LATER -DONLY_C_LOCALE=0 -DONNX_ML=1 -DONNX_NAMESPACE=onnx -DORT_ENABLE_STREAM -DPLATFORM_POSIX -DPROTOBUF_USE_DLLS -DUSE_CUDA=1 -DUSE_DX_INTEROP=0 -DUSE_FLASH_ATTENTION=1 -DUSE_FP8_KV_CACHE=1 -DUSE_MEMORY_EFFICIENT_ATTENTION=1 -D_GNU_SOURCE -Donnxruntime_providers_cuda_EXPORTS -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/onnxruntime-1.26.0/include/onnxruntime -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/onnxruntime-1.26.0/include/onnxruntime/core/session -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/pytorch_cpuinfo-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/onnxruntime-1.26.0/onnxruntime -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/gsl-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/date-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/onnx-src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/onnx-build -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/protobuf/3.21.9-b07b8f47dd1983d3cd0c0f051c28c6a1/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/flatbuffers-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/cutlass-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/cutlass-src/examples -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/cutlass-src/tools/util/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/cudnn_frontend-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/eigen3-src -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/safeint-src -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/cuda/13.3.0-46dd8459fe712b1803d2e4e8089fe749/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/cudnn/9.23.0.39-913fb4345828f9ceb331770b646c4256/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/mp11-src/include -DTHRUST_IGNORE_DEPRECATED_API -DCUB_IGNORE_DEPRECATED_API -Wno-deprecated-gpu-targets --static-global-template-stub=false -cudart shared -Xfatbin=-compress-all -compress-mode=size --expt-relaxed-constexpr --Werror default-stream-launch -Xcudafe "--diag_suppress=bad_friend_decl" -Xcudafe "--diag_suppress=unsigned_compare_with_zero" -Xcudafe "--diag_suppress=expr_has_no_effect" -O3 -DNDEBUG -std=c++20 "--generate-code=arch=compute_75,code=[sm_75]" "--generate-code=arch=compute_80,code=[sm_80]" "--generate-code=arch=compute_89,code=[sm_89]" "--generate-code=arch=compute_90a,code=[sm_90a]" "--generate-code=arch=compute_100a,code=[sm_100a]" "--generate-code=arch=compute_120a,code=[sm_120a]" -Xcompiler=-fPIC -Wno-deprecated-declarations -Xcudafe --diag_suppress=conversion_function_not_usable --compiler-options -Wall --compiler-options -Wno-deprecated-copy --compiler-options -Wno-nonnull-compare --compiler-options -Wno-interference-size -Xcompiler -Wno-nonnull-compare -Xcompiler -Wno-interference-size --threads 1 --diag-suppress=177 --static-global-template-stub=false --diag-suppress=221 -Xcompiler -Wno-attributes -Xcompiler -Wno-reorder -Xcompiler -Wno-error=sign-compare -Xptxas=-w -Werror all-warnings -MD -MT CMakeFiles/onnxruntime_providers_cuda.dir/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/onnxruntime-1.26.0/onnxruntime/core/providers/cuda/math/binary_elementwise_ops_impl.cu.o -MF CMakeFiles/onnxruntime_providers_cuda.dir/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/onnxruntime-1.26.0/onnxruntime/core/providers/cuda/math/binary_elementwise_ops_impl.cu.o.d -x cu -c /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/onnxruntime-1.26.0/onnxruntime/core/providers/cuda/math/binary_elementwise_ops_impl.cu -o CMakeFiles/onnxruntime_providers_cuda.dir/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/onnxruntime-1.26.0/onnxruntime/core/providers/cuda/math/binary_elementwise_ops_impl.cu.o
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:126:169: error: using template type parameter 'absl::lts_20250814::container_internal::IfRRef::AddPtr' after 'typename'
126 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, const &,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:126:223: error: using template type parameter 'absl::lts_20250814::container_internal::IfRRef::AddPtr' after 'typename'
126 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, const &,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:126:234: error: template argument 5 is invalid
126 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, const &,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:126:234: error: template argument 6 is invalid
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:126:236: error: template argument 1 is invalid
126 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, const &,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:135:169: error: using template type parameter 'absl::lts_20250814::container_internal::IfRRef::AddPtr' after 'typename'
135 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, &&, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:135:211: error: template argument 5 is invalid
135 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, &&, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:135:213: error: template argument 1 is invalid
135 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, &&, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:144:200: error: using template type parameter 'absl::lts_20250814::container_internal::IfRRef::AddPtr' after 'typename'
144 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, &&, const &, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:144:211: error: template argument 6 is invalid
144 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, &&, const &, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:144:213: error: template argument 1 is invalid
144 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, &&, const &, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:162:169: error: using template type parameter 'absl::lts_20250814::container_internal::IfRRef::AddPtr' after 'typename'
162 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, const &,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:162:223: error: using template type parameter 'absl::lts_20250814::container_internal::IfRRef::AddPtr' after 'typename'
162 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, const &,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:162:234: error: template argument 5 is invalid
162 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, const &,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:162:234: error: template argument 6 is invalid
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:162:236: error: template argument 1 is invalid
162 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, const &,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:171:169: error: using template type parameter 'absl::lts_20250814::container_internal::IfRRef::AddPtr' after 'typename'
171 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, &&, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:171:211: error: template argument 5 is invalid
171 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, &&, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:171:213: error: template argument 1 is invalid
171 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, &&, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:180:200: error: using template type parameter 'absl::lts_20250814::container_internal::IfRRef::AddPtr' after 'typename'
180 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, &&, const &, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:180:211: error: template argument 6 is invalid
180 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, &&, const &, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:180:213: error: template argument 1 is invalid
180 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, &&, const &, false,
| ^
ninja: build stopped: subcommand failed.
```
### Urgency
_No response_
### Target platform
RedHat Enterprise Linux
### Build script
```
+ cmake ../onnxruntime-1.26.0/cmake -G Ninja -Wno-dev -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/BUILDROOT/92eddeb1b3f2986c95279595476fcca1/opt/cmssw/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1 -DCMAKE_INSTALL_LIBDIR=lib -Donnxruntime_ENABLE_PYTHON=ON -Donnxruntime_BUILD_SHARED_LIB=ON -Donnxruntime_USE_CUDA=ON -Donnxruntime_CUDA_HOME=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/cuda/13.3.0-46dd8459fe712b1803d2e4e8089fe749 -Donnxruntime_CUDNN_HOME=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/cudnn/9.23.0.39-913fb4345828f9ceb331770b646c4256 -Donnxruntime_NVCC_THREADS=1 '-DCMAKE_CUDA_ARCHITECTURES=75;80;89;90;100;120' '-DCMAKE_CUDA_FLAGS=-DTHRUST_IGNORE_DEPRECATED_API -DCUB_IGNORE_DEPRECATED_API -Wno-deprecated-gpu-targets --static-global-template-stub=false -cudart shared' -DCMAKE_CUDA_RUNTIME_LIBRARY=Shared -DCMAKE_TRY_COMPILE_PLATFORM_VARIABLES=CMAKE_CUDA_RUNTIME_LIBRARY -Donnxruntime_BUILD_CSHARP=OFF -Donnxruntime_ENABLE_MICROSOFT_INTERNAL=OFF -Donnxruntime_USE_TENSORRT=OFF -Donnxruntime_CROSS_COMPILING=OFF -Donnxruntime_USE_FULL_PROTOBUF=ON -Donnxruntime_DISABLE_CONTRIB_OPS=OFF -Donnxruntime_BUILD_UNIT_TESTS=OFF '-DCMAKE_PREFIX_PATH=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/zlib/1.3.2-53c061acc706fe8f28fd9477d9981c20;/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/libpng/1.6.44-023e6fa0640b53f2ede9af9e3b636961;/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/protobuf/3.21.9-b07b8f47dd1983d3cd0c0f051c28c6a1;/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/py3-pybind11/3.0.1-40a12f1b4fe2393aef934ccb44fb2efc;/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/re2/2021-06-01-e9f9384affd92ef58191e1410d1e9a0d;/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/eigen/c1d637433e3b3f9012b226c2c9125c494b470ae6-98caed40e4d2e9af234831b4eafb854a' '-DCMAKE_CXX_FLAGS=-Wno-error=stringop-overflow -Wno-error=maybe-uninitialized -Wno-error=overloaded-virtual'
```
### Error / output
```
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:126:169: error: using template type parameter 'absl::lts_20250814::container_internal::IfRRef::AddPtr' after 'typename'
126 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, const &,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:126:223: error: using template type parameter 'absl::lts_20250814::container_internal::IfRRef::AddPtr' after 'typename'
126 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, const &,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:126:234: error: template argument 5 is invalid
126 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, const &,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:126:234: error: template argument 6 is invalid
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:126:236: error: template argument 1 is invalid
126 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, const &,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:135:169: error: using template type parameter 'absl::lts_20250814::container_internal::IfRRef::AddPtr' after 'typename'
135 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, &&, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:135:211: error: template argument 5 is invalid
135 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, &&, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:135:213: error: template argument 1 is invalid
135 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, &&, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:144:200: error: using template type parameter 'absl::lts_20250814::container_internal::IfRRef::AddPtr' after 'typename'
144 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, &&, const &, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:144:211: error: template argument 6 is invalid
144 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, &&, const &, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:144:213: error: template argument 1 is invalid
144 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, &&, const &, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:162:169: error: using template type parameter 'absl::lts_20250814::container_internal::IfRRef::AddPtr' after 'typename'
162 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, const &,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:162:223: error: using template type parameter 'absl::lts_20250814::container_internal::IfRRef::AddPtr' after 'typename'
162 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, const &,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:162:234: error: template argument 5 is invalid
162 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, const &,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:162:234: error: template argument 6 is invalid
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:162:236: error: template argument 1 is invalid
162 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, const &,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:171:169: error: using template type parameter 'absl::lts_20250814::container_internal::IfRRef::AddPtr' after 'typename'
171 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, &&, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:171:211: error: template argument 5 is invalid
171 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, &&, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:171:213: error: template argument 1 is invalid
171 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, const &, &&, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:180:200: error: using template type parameter 'absl::lts_20250814::container_internal::IfRRef::AddPtr' after 'typename'
180 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, &&, const &, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:180:211: error: template argument 6 is invalid
180 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, &&, const &, false,
| ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/onnxruntime/1.26.0-92eddeb1b3f2986c95279595476fcca1/build/_deps/abseil_cpp-src/absl/container/internal/raw_hash_map.h:180:213: error: template argument 1 is invalid
180 | ABSL_INTERNAL_X(insert_or_assign, insert_or_assign_impl, &&, const &, false,
| ^
```
### Visual Studio Version
_No response_
### GCC / Compiler Version
nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2026 NVIDIA Corporation Built on Fri_Apr_24_07:22:02_PM_PDT_2026 Cuda compilation tools, release 13.3, V13.3.33 Build cuda_13.3.r13.3/compiler.37862127_0
Contributor guide
Research direction
Reproduce the ONNX Runtime 1.26.0 build with CUDA 13.3.0 and inspect onnxruntime/core/providers/cuda/math/binary_elementwise_ops_impl.cu alongside the reported absl/container/internal/raw_hash_map.h errors. Compare the failing template instantiation with the build configuration and dependency versions; done means the CUDA provider builds successfully with this toolchain.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cmake, cpp
- Domain
- build-system, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100