microsoft / microsoft/onnxruntime-genai

Build for Android failing: error: no template named 'unordered_map' in namespace 'std'

Open
#1,291 7 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

platform:mobile
Dominant language
C++
Stars
1.1k
Forks
354
Avg merge
2d 16h
Merged PRs (30d)
85

Description

**Describe the bug**
When building onnxruntime-genai for Android on Windows using OpenJDK 21, NDK 25.2, and CMake 3.31.6, the build fails with errors ```error: no template named 'unordered_map' in namespace 'std'; did you mean 'nlohmann::ordered_map'?```
I also tried to build the project under Linux with JDK 17 and NDK 25.2 and got the same errors.

**ChatGPT problem idea:**
> The error message suggests a potential conflict with `nlohmann::ordered_map`. This issue appears to be caused by missing includes in the generated onnxruntime-extensions source files (for example, in `tokenizer_common.h` and `c_api_utils.hpp`).

**To Reproduce**
Steps to reproduce the behavior:
1. Clone the onnxruntime-genai repository.
2. Try to build the project:
```
python build.py --build_java --android --android_home "[...]\AppData\Local\Android\Sdk" --android_ndk_path "[...]\AppData\Local\Android\Sdk\ndk\25.2.9519653" --android_abi arm64-v8a --config Release
```
3. The build fails with errors:
```
error: no template named 'unordered_map' in namespace 'std'
```

**Expected behavior**
The build should complete successfully. The project should include the proper headers (e.g., ``) so that `std::unordered_map` is correctly recognized by the compiler.

**Desktop:**
- OS: Windows 11
- Compiler: Clang (Android NDK 25.2)
- CMake: 3.31.6

**Relevant Log**
```
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: [...]/AppData/Local/Android/Sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/windows-x86_64/bin/clang++.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using ONNX Runtime package Microsoft.ML.OnnxRuntime version 1.20.0-dev-20241007-1101-407c1ab2e2
-- ORT_HEADER_DIR: [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/ortlib-src/build/native/include
-- ORT_LIB_DIR: [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/ortlib-src/runtimes/android/native/jni/arm64-v8a
Loading Dependencies URLs ...
Loading Dependencies ...
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD

[...]

[131/208] Building C object _deps/onnxruntime_extensions-build/CMakeFiles/libpng_static_c.dir/__/dlib-src/dlib/external/zlib/inflate.c.o
warning: unknown warning option '-Wno-deprecated-non-prototype'; did you mean '-Wno-deprecated-coroutine'? [-Wunknown-warning-option]
1 warning generated.
[158/208] Building CXX object _deps/onnxruntime_extensions-build/CMakeFiles/ocos_operators.dir/operators/tokenizer/bpe_kernels.cc.o
FAILED: _deps/onnxruntime_extensions-build/CMakeFiles/ocos_operators.dir/operators/tokenizer/bpe_kernels.cc.o
[...]\AppData\Local\Android\Sdk\ndk\25.2.9519653\toolchains\llvm\prebuilt\windows-x86_64\bin\clang++.exe --target=aarch64-none-linux-android27 --sysroot=[...]/AppData/Local/Android/Sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/windows-x86_64/sysroot -DENABLE_C_API -DENABLE_DLIB -DENABLE_DR_LIBS -DENABLE_GPT2_TOKENIZER -DENABLE_MATH -DENABLE_TOKENIZER -DENABLE_VISION -DJSON_HAS_CPP_17=1 -DOCOS_ENABLE_VENDOR_IMAGE_CODECS -DOCOS_SHARED_LIBRARY -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/ortlib-src/build/native/include -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/gsl-src/include -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/include -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/include/custom_op -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/base -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/dlib-src -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/dlib-src/dlib/external/libpng -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/dlib-src/dlib/external/libjpeg -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/nlohmann_json-src/include -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/dlib-src/dlib/external/zlib -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/dr_libs-src -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -O3 -DNDEBUG -std=c++17 -fPIC -MD -MT _deps/onnxruntime_extensions-build/CMakeFiles/ocos_operators.dir/operators/tokenizer/bpe_kernels.cc.o -MF _deps\onnxruntime_extensions-build\CMakeFiles\ocos_operators.dir\operators\tokenizer\bpe_kernels.cc.o.d -o _deps/onnxruntime_extensions-build/CMakeFiles/ocos_operators.dir/operators/tokenizer/bpe_kernels.cc.o -c [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.cc
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.cc:9:
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.h:11:
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/tokenizer_common.h:38:23: error: no template named 'unordered_map' in namespace 'std'; did you mean 'nlohmann::ordered_map'?
using AddedTokenMap = std::unordered_map;
^~~~~~~~~~~~~~~~~~
nlohmann::ordered_map
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/nlohmann_json-src/include/nlohmann/json_fwd.hpp:67:8: note: 'nlohmann::ordered_map' declared here
struct ordered_map;
^
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.cc:9:
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.h:11:
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/tokenizer_common.h:38:28: error: too few template arguments for class template 'ordered_map'
using AddedTokenMap = std::unordered_map;
^
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/nlohmann_json-src/include/nlohmann/json_fwd.hpp:67:8: note: template is declared here
struct ordered_map;
^
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.cc:9:
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.h:28:41: error: no type named 'AddedTokenMap' in namespace 'ort_extensions'; did you mean 'AddedToken'?
using AddedTokenMap = ort_extensions::AddedTokenMap;
~~~~~~~~~~~~~~~~^~~~~~~~~~~~~
AddedToken
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/tokenizer_common.h:20:8: note: 'AddedToken' declared here
struct AddedToken final {
^
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.cc:10:
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/tokenizer_jsconfig.hpp:248:3: error: unknown type name 'AddedTokenMap'; did you mean 'CLIPTokenizer::AddedTokenMap'?
AddedTokenMap added_tokens_;
^~~~~~~~~~~~~
CLIPTokenizer::AddedTokenMap
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.h:28:9: note: 'CLIPTokenizer::AddedTokenMap' declared here
using AddedTokenMap = ort_extensions::AddedTokenMap;
^
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.cc:10:
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/tokenizer_jsconfig.hpp:284:23: error: no member named 'emplace' in 'ort_extensions::AddedToken'
added_tokens_.emplace(ustring(tok_extended.content_), tok_extended);
~~~~~~~~~~~~~ ^
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.cc:11:
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_tokenizer_model.hpp:313:30: error: unknown type name 'AddedTokenMap'; did you mean 'CLIPTokenizer::AddedTokenMap'?
void LoadAddedTokens(const AddedTokenMap& added_tokens) {
^~~~~~~~~~~~~
CLIPTokenizer::AddedTokenMap
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.h:28:9: note: 'CLIPTokenizer::AddedTokenMap' declared here
using AddedTokenMap = ort_extensions::AddedTokenMap;
^
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.cc:11:
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_tokenizer_model.hpp:323:70: error: unknown type name 'AddedTokenMap'; did you mean 'CLIPTokenizer::AddedTokenMap'?
bpe::TokenPairs SplitByAddedAndSpecial(const ustring& input, const AddedTokenMap& t_map) const {
^~~~~~~~~~~~~
CLIPTokenizer::AddedTokenMap
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.h:28:9: note: 'CLIPTokenizer::AddedTokenMap' declared here
using AddedTokenMap = ort_extensions::AddedTokenMap;
^
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.cc:11:
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_tokenizer_model.hpp:314:35: error: invalid range expression of type 'const CLIPTokenizer::AddedTokenMap' (aka 'const ort_extensions::AddedToken'); no viable 'begin' function available
for (const auto& [key, token] : added_tokens) {
^ ~~~~~~~~~~~~
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_tokenizer_model.hpp:337:40: error: no member named 'find' in 'ort_extensions::AddedToken'
auto iter_tok_extend = t_map.find(std::u32string(token));
~~~~~ ^
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_tokenizer_model.hpp:338:40: error: no member named 'end' in 'ort_extensions::AddedToken'
if (iter_tok_extend != t_map.end()) {
~~~~~ ^
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.cc:681:21: error: no member named 'emplace' in 'ort_extensions::AddedToken'
added_tokens_.emplace(ustring(tok_extended.content_), tok_extended);
~~~~~~~~~~~~~ ^
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.cc:693:43: error: no member named 'find' in 'ort_extensions::AddedToken'
auto existing_token = added_tokens_.find(ustring(token.value("content", "")));
~~~~~~~~~~~~~ ^
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.cc:694:43: error: no member named 'end' in 'ort_extensions::AddedToken'
if (existing_token == added_tokens_.end()) { // Token doesn't exist yet
~~~~~~~~~~~~~ ^
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.cc:706:23: error: no member named 'emplace' in 'ort_extensions::AddedToken'
added_tokens_.emplace(ustring(added_token.content_), added_token);
~~~~~~~~~~~~~ ^
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.cc:712:39: error: invalid range expression of type 'KernelBpeTokenizer::AddedTokenMap' (aka 'ort_extensions::AddedToken'); no viable 'begin' function available
for (const auto& [key, added_token] : added_tokens_) {
^ ~~~~~~~~~~~~~
15 errors generated.
[167/208] Building CXX object _deps/onnxruntime_extensions-build/CMakeFiles/ocos_operators.dir/operators/tokenizer/tokenizers.cc.o
FAILED: _deps/onnxruntime_extensions-build/CMakeFiles/ocos_operators.dir/operators/tokenizer/tokenizers.cc.o
[...]\AppData\Local\Android\Sdk\ndk\25.2.9519653\toolchains\llvm\prebuilt\windows-x86_64\bin\clang++.exe --target=aarch64-none-linux-android27 --sysroot=[...]/AppData/Local/Android/Sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/windows-x86_64/sysroot -DENABLE_C_API -DENABLE_DLIB -DENABLE_DR_LIBS -DENABLE_GPT2_TOKENIZER -DENABLE_MATH -DENABLE_TOKENIZER -DENABLE_VISION -DJSON_HAS_CPP_17=1 -DOCOS_ENABLE_VENDOR_IMAGE_CODECS -DOCOS_SHARED_LIBRARY -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/ortlib-src/build/native/include -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/gsl-src/include -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/include -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/include/custom_op -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/base -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/dlib-src -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/dlib-src/dlib/external/libpng -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/dlib-src/dlib/external/libjpeg -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/nlohmann_json-src/include -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/dlib-src/dlib/external/zlib -I[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/dr_libs-src -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -O3 -DNDEBUG -std=c++17 -fPIC -MD -MT _deps/onnxruntime_extensions-build/CMakeFiles/ocos_operators.dir/operators/tokenizer/tokenizers.cc.o -MF _deps\onnxruntime_extensions-build\CMakeFiles\ocos_operators.dir\operators\tokenizer\tokenizers.cc.o.d -o _deps/onnxruntime_extensions-build/CMakeFiles/ocos_operators.dir/operators/tokenizer/tokenizers.cc.o -c [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/tokenizers.cc
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/tokenizers.cc:7:
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.h:11:
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/tokenizer_common.h:38:23: error: no template named 'unordered_map' in namespace 'std'; did you mean 'nlohmann::ordered_map'?
using AddedTokenMap = std::unordered_map;
^~~~~~~~~~~~~~~~~~
nlohmann::ordered_map
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/nlohmann_json-src/include/nlohmann/json_fwd.hpp:67:8: note: 'nlohmann::ordered_map' declared here
struct ordered_map;
^
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/tokenizers.cc:7:
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.h:11:
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/tokenizer_common.h:38:28: error: too few template arguments for class template 'ordered_map'
using AddedTokenMap = std::unordered_map;
^
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/nlohmann_json-src/include/nlohmann/json_fwd.hpp:67:8: note: template is declared here
struct ordered_map;
^
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/tokenizers.cc:7:
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.h:28:41: error: no type named 'AddedTokenMap' in namespace 'ort_extensions'; did you mean 'AddedToken'?
using AddedTokenMap = ort_extensions::AddedTokenMap;
~~~~~~~~~~~~~~~~^~~~~~~~~~~~~
AddedToken
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/tokenizer_common.h:20:8: note: 'AddedToken' declared here
struct AddedToken final {
^
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/tokenizers.cc:8:
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_tokenizer_model.hpp:313:30: error: unknown type name 'AddedTokenMap'; did you mean 'CLIPTokenizer::AddedTokenMap'?
void LoadAddedTokens(const AddedTokenMap& added_tokens) {
^~~~~~~~~~~~~
CLIPTokenizer::AddedTokenMap
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.h:28:9: note: 'CLIPTokenizer::AddedTokenMap' declared here
using AddedTokenMap = ort_extensions::AddedTokenMap;
^
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/tokenizers.cc:8:
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_tokenizer_model.hpp:323:70: error: unknown type name 'AddedTokenMap'; did you mean 'CLIPTokenizer::AddedTokenMap'?
bpe::TokenPairs SplitByAddedAndSpecial(const ustring& input, const AddedTokenMap& t_map) const {
^~~~~~~~~~~~~
CLIPTokenizer::AddedTokenMap
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.h:28:9: note: 'CLIPTokenizer::AddedTokenMap' declared here
using AddedTokenMap = ort_extensions::AddedTokenMap;
^
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/tokenizers.cc:8:
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_tokenizer_model.hpp:314:35: error: invalid range expression of type 'const CLIPTokenizer::AddedTokenMap' (aka 'const ort_extensions::AddedToken'); no viable 'begin' function available
for (const auto& [key, token] : added_tokens) {
^ ~~~~~~~~~~~~
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_tokenizer_model.hpp:337:40: error: no member named 'find' in 'ort_extensions::AddedToken'
auto iter_tok_extend = t_map.find(std::u32string(token));
~~~~~ ^
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_tokenizer_model.hpp:338:40: error: no member named 'end' in 'ort_extensions::AddedToken'
if (iter_tok_extend != t_map.end()) {
~~~~~ ^
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/tokenizers.cc:10:
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/tokenizer_op_impl.hpp:7:
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/ugm_kernels.hpp:24:
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/tokenizer_jsconfig.hpp:248:3: error: unknown type name 'AddedTokenMap'; did you mean 'CLIPTokenizer::AddedTokenMap'?
AddedTokenMap added_tokens_;
^~~~~~~~~~~~~
CLIPTokenizer::AddedTokenMap
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/bpe_kernels.h:28:9: note: 'CLIPTokenizer::AddedTokenMap' declared here
using AddedTokenMap = ort_extensions::AddedTokenMap;
^
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/tokenizers.cc:10:
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/tokenizer_op_impl.hpp:7:
In file included from [...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/ugm_kernels.hpp:24:
[...]/StudioProjects/onnxruntime-genai/build/Android/Release/_deps/onnxruntime_extensions-src/operators/tokenizer/tokenizer_jsconfig.hpp:284:23: error: no member named 'emplace' in 'ort_extensions::AddedToken'
added_tokens_.emplace(ustring(tok_extended.content_), tok_extended);
~~~~~~~~~~~~~ ^
10 errors generated.
[171/208] Building CXX object _deps/onnxruntime_extensions-build/CMakeFiles/ocos_operators.dir/operators/math/math.cc.o
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "[...]\StudioProjects\onnxruntime-genai\build.py", line 662, in
build(arguments, environment)
File "[...]\StudioProjects\onnxruntime-genai\build.py", line 587, in build
util.run(make_command, env=env)
File "[...]\StudioProjects\onnxruntime-genai\tools\python\util\run.py", line 56, in run
completed_process = subprocess.run(
File "[...]\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 524, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['C:\\Program Files\\CMake\\bin\\cmake.EXE', '--build', '[...]\\StudioProjects\\onnxruntime-genai\\build\\Android\\Release', '--config', 'Release']' returned non-zero exit status 1.
```

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with build.py and reproduce the Android build using the reported NDK, CMake, and ABI settings. Inspect the onnxruntime-extensions tokenizer sources, especially tokenizer_common.h, c_api_utils.hpp, bpe_kernels.cc, and the related headers, then confirm the build completes successfully for Android.

Written by the indexing model from the issue text.

Assessment

Tech stack
android, cmake, cpp, python
Domain
build-system, mobile
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.