manticoresoftware / manticoresoftware/columnar

Crash at insert with auto embeddings and html_strip

Open
#125 2 comments 0 reactions 1 assignee View on GitHub

@sanikolaev is already working on this.

Since Jan 8, 2026.

bug feature::vector_search
Dominant language
C++
Stars
110
Forks
23
Avg merge
3d 5h
Merged PRs (30d)
8

Description

### Bug Description:

```
drop table if exists t;
CREATE TABLE IF NOT EXISTS t (title text, embedding_vector FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='cosine' MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2' FROM='title') html_strip='1';
INSERT INTO t (title) VALUES ('Hello. I. Myâ¦');
```

causes:
```
Trying boost backtrace:
0# sphBacktrace(int, bool) in /opt/homebrew/Cellar/manticoresearch-dev/16.3.2-26010809-ec8034be1/bin/searchd
1# HandleCrash(int) in /opt/homebrew/Cellar/manticoresearch-dev/16.3.2-26010809-ec8034be1/bin/searchd
2# _sigtramp in /usr/lib/system/libsystem_platform.dylib
3# pthread_kill in /usr/lib/system/libsystem_pthread.dylib
4# abort in /usr/lib/system/libsystem_c.dylib
5# _RNvCsiGVaDesi5rv_7___rustc26___rust_alloc_error_handler in /opt/homebrew/Cellar/manticore-columnar-lib-dev/9.0.2B25122309-5b73f8d1/share/manticore/modules/lib_manticore_knn_embeddings.so
6# core::cell::once::OnceCell$LT$T$GT$::try_init::h5ae72910d44ba69b in /opt/homebrew/Cellar/manticore-columnar-lib-dev/9.0.2B25122309-5b73f8d1/share/manticore/modules/lib_manticore_knn_embeddings.so
7# std::panicking::panic_count::is_zero_slow_path::h0e49bc980a448302 in /opt/homebrew/Cellar/manticore-columnar-lib-dev/9.0.2B25122309-5b73f8d1/share/manticore/modules/lib_manticore_knn_embeddings.so
8# std::panicking::panic_with_hook::h93c775fc227522dd in /opt/homebrew/Cellar/manticore-columnar-lib-dev/9.0.2B25122309-5b73f8d1/share/manticore/modules/lib_manticore_knn_embeddings.so
9# std::panicking::panic_handler::_$u7b$$u7b$closure$u7d$$u7d$::h8561e58c9391724a in /opt/homebrew/Cellar/manticore-columnar-lib-dev/9.0.2B25122309-5b73f8d1/share/manticore/modules/lib_manticore_knn_embeddings.so
10# std::panicking::panic_handler::_$u7b$$u7b$closure$u7d$$u7d$::h8561e58c9391724a in /opt/homebrew/Cellar/manticore-columnar-lib-dev/9.0.2B25122309-5b73f8d1/share/manticore/modules/lib_manticore_knn_embeddings.so
11# std::sys::backtrace::__rust_end_short_backtrace::h55629ed76aaa9d26 in /opt/homebrew/Cellar/manticore-columnar-lib-dev/9.0.2B25122309-5b73f8d1/share/manticore/modules/lib_manticore_knn_embeddings.so
12# core::panicking::panic::h4a3c64c65dc98033 in /opt/homebrew/Cellar/manticore-columnar-lib-dev/9.0.2B25122309-5b73f8d1/share/manticore/modules/lib_manticore_knn_embeddings.so
13# alloc::raw_vec::RawVecInner$LT$A$GT$::reserve::do_reserve_and_handle::h076dffaa2fb6fbdc in /opt/homebrew/Cellar/manticore-columnar-lib-dev/9.0.2B25122309-5b73f8d1/share/manticore/modules/lib_manticore_knn_embeddings.so
14# alloc::raw_vec::handle_error::ha045b2c2db1a4ffe in /opt/homebrew/Cellar/manticore-columnar-lib-dev/9.0.2B25122309-5b73f8d1/share/manticore/modules/lib_manticore_knn_embeddings.so
15# core::panicking::assert_failed::h4b7885290ae7fea7 in /opt/homebrew/Cellar/manticore-columnar-lib-dev/9.0.2B25122309-5b73f8d1/share/manticore/modules/lib_manticore_knn_embeddings.so
16# _$LT$tokenizers..normalizers..NormalizerWrapper$u20$as$u20$tokenizers..tokenizer..Normalizer$GT$::normalize::h2c05c1a59a04c0aa in /opt/homebrew/Cellar/manticore-columnar-lib-dev/9.0.2B25122309-5b73f8d1/share/manticore/modules/lib_manticore_knn_embeddings.so
17# tokenizers::tokenizer::TokenizerImpl$LT$M$C$N$C$PT$C$PP$C$D$GT$::encode_single_sequence::_$u7b$$u7b$closure$u7d$$u7d$::h4495171d93cc7de2 in /opt/homebrew/Cellar/manticore-columnar-lib-dev/9.0.2B25122309-5b73f8d1/share/manticore/modules/lib_manticore_knn_embeddings.so
18# tokenizers::tokenizer::TokenizerImpl$LT$M$C$N$C$PT$C$PP$C$D$GT$::encode_single_sequence::hd0c7e3d5319f73b9 in /opt/homebrew/Cellar/manticore-columnar-lib-dev/9.0.2B25122309-5b73f8d1/share/manticore/modules/lib_manticore_knn_embeddings.so
19# _$LT$manticore_knn_embeddings..model..Model$u20$as$u20$manticore_knn_embeddings..model..TextModel$GT$::predict::hd42ecbec4431f294 in /opt/homebrew/Cellar/manticore-columnar-lib-dev/9.0.2B25122309-5b73f8d1/share/manticore/modules/lib_manticore_knn_embeddings.so
20# manticore_knn_embeddings::model::text_model_wrapper::TextModelWrapper::make_vect_embeddings::h37a555da2ec00bee in /opt/homebrew/Cellar/manticore-columnar-lib-dev/9.0.2B25122309-5b73f8d1/share/manticore/modules/lib_manticore_knn_embeddings.so
21# knn::TextToEmbeddings_c::Convert(std::__1::vector>, std::__1::allocator>>> const&, std::__1::vector>, std::__1::allocator>>>&, std::__1::basic_string, std::__1::allocator>&) const in /opt/homebrew/Cellar/manticore-columnar-lib-dev/9.0.2B25122309-5b73f8d1/share/manticore/modules/lib_manticore_knn.so
22# RtAccum_t::GenerateEmbeddings(int, int, sph::Vector_T, sph::DefaultRelimit, sph::DefaultStorage_T> const&, std::__1::vector>, std::__1::allocator>>>, std::__1::allocator>, std::__1::allocator>>>>>&, columnar::Iterator_i*, CSphString&) in /opt/homebrew/Cellar/manticoresearch-dev/16.3.2-26010809-ec8034be1/bin/searchd
23# RtAccum_t::FetchEmbeddings(TableEmbeddings_c*, sph::Vector_T, sph::DefaultRelimit, sph::DefaultStorage_T> const&, CSphString&) in /opt/homebrew/Cellar/manticoresearch-dev/16.3.2-26010809-ec8034be1/bin/searchd
24# RtIndex_c::Commit(int*, RtAccum_t*, CSphString*) in /opt/homebrew/Cellar/manticoresearch-dev/16.3.2-26010809-ec8034be1/bin/searchd
25# CommitMonitor_c::CommitNonEmptyCmds(RtIndex_i*, ReplicationCommand_t const&, bool) const in /opt/homebrew/Cellar/manticoresearch-dev/16.3.2-26010809-ec8034be1/bin/searchd
26# CommitMonitor_c::Commit() in /opt/homebrew/Cellar/manticoresearch-dev/16.3.2-26010809-ec8034be1/bin/searchd
27# HandleCmdReplicateImpl(RtAccum_t&, int*, CSphString*, int*) in /opt/homebrew/Cellar/manticoresearch-dev/16.3.2-26010809-ec8034be1/bin/searchd
28# sphHandleMysqlInsert(StmtErrorReporter_i&, SqlStmt_t const&) in /opt/homebrew/Cellar/manticoresearch-dev/16.3.2-26010809-ec8034be1/bin/searchd
29# ClientSession_c::Execute(std::__1::pair, RowBuffer_i&) in /opt/homebrew/Cellar/manticoresearch-dev/16.3.2-26010809-ec8034be1/bin/searchd
30# SqlServe(std::__1::unique_ptr>) in /opt/homebrew/Cellar/manticoresearch-dev/16.3.2-26010809-ec8034be1/bin/searchd
31# MultiServe(std::__1::unique_ptr>, std::__1::pair, Proto_e) in /opt/homebrew/Cellar/manticoresearch-dev/16.3.2-26010809-ec8034be1/bin/searchd
32# std::__1::__function::__func, void ()>::operator()() in /opt/homebrew/Cellar/manticoresearch-dev/16.3.2-26010809-ec8034be1/bin/searchd
33# Threads::CoRoutine_c::CreateContext(std::__1::function, std::__1::pair)::'lambda'(boost::context::detail::transfer_t)::__invoke(boost::context::detail::transfer_t) in /opt/homebrew/Cellar/manticoresearch-dev/16.3.2-26010809-ec8034be1/bin/searchd
```

### Manticore Search Version:

Manticore 16.3.2 ec8034be1@26010809 dev (columnar 9.0.2 5b73f8d@25122308) (secondary 9.0.2 5b73f8d@25122308) (knn 9.0.2 5b73f8d@25122308) (embeddings 1.0.1)

### Operating System Version:

macos

### Have you tried the latest development version?

No

### Internal Checklist:

To be completed by the assignee. Check off tasks that have been completed or are not applicable.

- [x] Implementation completed
- [ ] Tests developed
- [x] Documentation updated
- [x] Documentation reviewed

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.