apple / apple/foundationdb

StorageServer processes likely to see OutOfMemory error after storage engine change

Open
#6,431 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

Due to the way different storage engines allocate memory, there is a high risk of running out of memory shortly after one storage engine is closed and another is opened.

The Memory, SQLite, and Redwood storage engines all use different `FastAllocator` sizes for the bulk of their memory usage, so when switching from any of these engines to anything else much of the memory they were using will not be freed and instead remain in the FastAllocator magazines.

While there are several ways to address this, the easiest way is for a storage server process to exit after closing a storage engine of one type and creating one of another type.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the StorageServer path that closes one storage engine and creates another, focusing on the Memory, SQLite, and Redwood engine transitions described in the issue. Verify the process-exit behavior after an engine switch and confirm that the change prevents memory from remaining allocated in FastAllocator magazines.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.