lightninglabs / lightninglabs/taproot-assets

[enhancement]: optimize performance of DB: batched writes (sqlite), concurrent writes (postgres)

Open
#417 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

database postgres sqlite
Dominant language
Go
Stars
525
Forks
150
Avg merge
2d 15h
Merged PRs (30d)
31

Description

### Background
---
To prevent previous dysfunction,
write locks were added in https://github.com/lightninglabs/taproot-assets/commit/b190d32087a3d579fbfca4cd06548d59d0b46694#diff-34ab0b3a4593407088a7516e921f22607e4237f236325db502804e33433327b2R303
which ultimately won't be necessary once #261 is fixed. After which,
**DB writes can be optimized with batched writes (sqlite) or concurrent writes (postgres)**

### Potential parameters to be chose / Design criteria
---
Given sqlite, at what which point should the batch write cut off e.g. How long should write requests be collected for the ultimate batched write?

Given postgres,
- TANSTAAFL, What are the fundamental tradeoffs with concurrent writes?
- Given concurrent writes, there are possibly writes occurring when other actions are being taken by the daemon, how should the consistency be addressed after certain data has been timestamped by the bitcoin wallet backend.

What interactions are there with the MS-SMT tree level structure and SIMD processing that we might want to leverage for highly paralizable pocket universe load balancer design architectures?

**Related issues:**
https://github.com/lightninglabs/taproot-assets/issues/374

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the write-lock change in commit b190d320 and the context in issues #261 and #374. Investigate batching for SQLite and concurrent writes for PostgreSQL, including the stated consistency and timestamping concerns. Done should include a justified design and validated performance improvements for both database backends.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, postgresql, sqlite
Domain
databases, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.