apple / apple/foundationdb

Multi-version clients can erase each others trace files

Open
#675 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

When trace files are opened or rolled, we run some code to clean up old trace files from the process opening a new file. When using the multi-version client, this can erase files from other versions of the client that are running in the same process.

Normally, I'm not sure this is a huge deal because the trace files from the client libs are probably similar in size, with the active client possibly being marginally larger. However, if we run into an issue where one client lib is writing a lot more log output than the others, it's possible that it could end up deleting all of the logs for the other client libs.

I'm not sure exactly what the right thing to do here is yet. We could let each library treat its log files separately, but then we would be writing some multiple of the amount of logs we specified. We could also evenly split the allowed log space between the libs or possibly have some minimum retention per lib, leaving the rest up for grabs.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the code that opens or rolls trace files and cleans up old files, then examine how multi-version clients share that process and log space. Done means one client version cannot erase another version's trace files while the configured retention limit is still respected.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
observability-sre
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.