JuliaData / JuliaData/MemPool.jl
Add transactions
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 25
- Forks
- 16
- Avg merge
- 4h 56m
- Merged PRs (30d)
- 1
Description
At the moment, our storage layer (in StorageState) is non-transactional, and so it's not possible to ensure that a certain set of operations occur without other (incompatible or performance-degrading) operations occurring in-between. This can make it impossible for the SimpleRecencyAllocator (which provides our swap-to-disk functionality) to be able to provide any guarantees that memory/disk usage actually falls within user-provided limits.
The obvious solution here is to add "transactions", where the set of operations to be performed in sequence are batched up and submitted as a single logical operation. Any transaction in progress can exclude other operations or transactions from occurring on the same piece(s) of data until the current transaction completes. Additionally, for performance, we can implement some logic to "merge" compatible operations across multiple transactions (or within the same transaction) when the end result will be identical to running the operations exclusively.
@dbedi3311
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the StorageState storage layer and the SimpleRecencyAllocator swap-to-disk component. Clarify the transaction API, exclusion semantics, and compatible-operation merging with maintainers before implementation; done should include guarantees that memory and disk usage remain within user-provided limits.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100