baidu / baidu/tera

OOM may occur on memtable

Open
#519 0 comments 0 reactions 0 assignees View on GitHub
class/bug
Dominant language
C++
Stars
1.9k
Forks
432
PR merge metrics
No merged PRs in 30d

Description

lg存储属性设置为memory时,持续写入可能导致tabletnode出现oom.

最简单的方案就是限制单个tablet对应的这段区间,LG(storage=memory)的大小。超过设定的阈值以后,只能删,不能再写。

这个阈值可以对应多种级别,例如tabletnode级、tablet级或lg级。
目前将这个阈值对应到lg,这也和每个lg对应一个存储引擎,有自己独立的memtable相合。毕竟,当单个tabletnode上有太多tablet或者lg时,问题不出在内存表这里。

Contributor guide

No contributing guide indexed for this repository

Research direction

No files, tests, or entry points are named in the issue. Begin by locating the implementation of memory-backed LG storage and its memtable accounting, then determine how an LG-level size threshold should block further writes while allowing deletes. Done means sustained writes no longer cause OOM when the configured threshold is exceeded.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.