Do we want/need compaction callbacks?
Open
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 732
- Forks
- 67
- PR merge metrics
- No merged PRs in 30d
Description
This is addressing some FIXMEs like the following. Should be provide the application with these in general?
36 // FIXME: [yfogel 2020-01-11] Need to add (to both merge_tuple fns)
37 // bool is_query (or enum)
38 // - Application needs to know if this is a compaction because that means
39 // that after merging data some of that info goes away permanently.
40 // e.g. if it needs to deallocate space.
41 // void* context:
42 // The above can be used to:
43 // - get message_size (in case we need to memmove parts that don't get updated)
44 // - let app keep track of statistics
45 // - let app do potentially heavier weight stuff (log things that need to be
46 // deallocated/learn whether blind inserts/deletes actually did anything)
47 // int thread_id: (0..n-1 as opposed to process id)
48 // - Any writing that app does with context can be more performant by
49 // providing it the thread id.
50 // As part of above we *may* need to add two new callbacks:
51 // start_compaction
52 // end_compaction
53 // In case the application needs to log/commit things to deallocate.
54 // It's not obvious it's necessary because even with the above, the app
55 // needs to deal with the same keys to be compacted twice sinlce
56 // logging & recovery is not combined between app & splinter.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the FIXME describing both merge_tuple functions and review how compaction and application callbacks currently interact. Determine whether compaction state, context, message size, thread ID, and start_compaction/end_compaction callbacks are needed, then document a resolved callback API and its completion criteria.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100