alibaba / alibaba/havenask

索引数据挂载在分布式存储searcher无法正常启动

Open
#38 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
1.8k
Forks
352
PR merge metrics
No merged PRs in 30d

Description

将构建好的索引数据挂载到分布式存储后searcher一直无法成功启动,查看ha3.log发现如下ERROR:
[ERROR] [201, build/release64/indexlib/file_system/mmap_file_node.cpp -- LoadData():309] [lock file:[/home/xxx/cases/normal/workdir/local_search_12000/in0_0/runtimedata/in0/generation_0/partition_0_65535/segment_14_level_0/index/title/bitmap_posting] FAILED, errno: 12, errmsg: Cannot allocate memory]
查看代码是mmap_file_node.cpp中的mlock报错
~~~
if (mlock(base + offset, lockLen) < 0)
{
INDEXLIB_FATAL_ERROR(FileIO, "lock file: [%s] FAILED"
", errno: %d, errmsg: %s",
GetPath().c_str(), errno, strerror(errno));
}
~~~
查看mlock相关文档时认为是
ENOMEM (mlock(), mlock2(), and munlock()) Some of the specified
address range does not correspond to mapped pages in the
address space of the process.
这个问题导致的,目前通过cp的方式将数据拷贝进来后searcher才能过成功启动。
请问这个问题有没有更好的解决方式呢?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the ERROR in ha3.log and inspect LoadData() in build/release64/indexlib/file_system/mmap_file_node.cpp, especially the mlock call and reported errno. Reproduce searcher startup with the index mounted from distributed storage, compare it with the successful cp-based setup, and determine a reliable resolution that avoids the workaround.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.