索引数据挂载在分布式存储searcher无法正常启动
- Dominant language
- C++
- Stars
- 1.8k
- Forks
- 352
- PR merge metrics
- No merged PRs in 30d
Description
将构建好的索引数据挂载到分布式存储后searcher一直无法成功启动,查看ha3.log发现如下ERROR:
[ERROR] [201, build/release64/indexlib/file_system/mmap_file_node.cpp -- LoadData():309] [lock file:[/home/xxx/cases/normal/workdir/local_search_12000/in0_0/runtimedata/in0/generation_0/partition_0_65535/segment_14_level_0/index/title/bitmap_posting] FAILED, errno: 12, errmsg: Cannot allocate memory]
查看代码是mmap_file_node.cpp中的mlock报错
~~~
if (mlock(base + offset, lockLen) < 0)
{
INDEXLIB_FATAL_ERROR(FileIO, "lock file: [%s] FAILED"
", errno: %d, errmsg: %s",
GetPath().c_str(), errno, strerror(errno));
}
~~~
查看mlock相关文档时认为是
ENOMEM (mlock(), mlock2(), and munlock()) Some of the specified
address range does not correspond to mapped pages in the
address space of the process.
这个问题导致的,目前通过cp的方式将数据拷贝进来后searcher才能过成功启动。
请问这个问题有没有更好的解决方式呢?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the ERROR in ha3.log and inspect LoadData() in build/release64/indexlib/file_system/mmap_file_node.cpp, especially the mlock call and reported errno. Reproduce searcher startup with the index mounted from distributed storage, compare it with the successful cp-based setup, and determine a reliable resolution that avoids the workaround.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100