apple / apple/foundationdb

Revisit the TLog's 2GB queue size

Open
#620 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

Our transaction logs currently start spilling from the disk queue into a b-tree if the data contained within memory exceeds 1.5GB. This is a decision that was made about 7 years ago, and can likely be revisited given the changes in average expected hardware.

It'd also be worth looking at cloud offerings and how we would recommend deployment work to set reasonable defaults. AWS Large has 8GB of RAM, but the largest 1 core VM (to keep the disk:cpu ratio low) would be a Small with 2GB of RAM, making 1.5GB a reasonable default.

Contributor guide

Open the contributing guide

Research direction

Start by tracing where the transaction log switches from the disk queue to a b-tree at the 1.5GB threshold. Compare that behavior with the cited AWS hardware constraints and determine a justified default and deployment recommendation; done means an agreed change or documented rationale.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.