tursodatabase / tursodatabase/libsql

Support changing the encryption key in libsql-server

Open
#976 12 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

encryption at rest enhancement
Dominant language
C
Stars
17.2k
Forks
531
Avg merge
1h 12m
Merged PRs (30d)
1

Description

With encryption enabled, we would like to be able to change the encryption key. This operation is supported in SQLite3MultipleCiphers and called rekey. Important things to take into account:

  1. Rekeying is not supported for a database in WAL mode, as mentioned here in a slightly scary message: https://github.com/utelle/SQLite3MultipleCiphers/blob/main/CHANGELOG.md#changed-34
  2. Rekeying operation in SQLite3MultipleCiphers is based on setting up a new encryption key and running vacuum.
  3. Vacuum is an operation that rewrites the database to a new file, purging unused free pages along the way.

Taking all the above into account, a rekeying operation can be safely implemented as follows, with incurred downtime:

  1. Stop serving user requests to stop reads and writes from happening
  2. Checkpoint the database in TRUNCATE mode - that leaves us with no WAL file, no WAL frames
  3. Switch the journaling mode to the regular journal, by calling PRAGMA journal_mode=delete
  4. Now that we're not in WAL mode, it's safe to call PRAGMA rekey=new_key (or in our case, the C API equivalent sqlite3_rekey). That will perform a VACUUM while also encrypting all data with new_key
  5. Switch the journaling mode back to WAL with PRAGMA journal_mode=wal
  6. Resume serving user requests, all data is now encrypted with new_key

An alternative solution would be to try and bring back support for rekeying in WAL mode, but there are 3 downsides to consider:

  1. Rekeying was disabled after reportedly causing db corruption in WAL mode, probably because VACUUM in WAL mode has a different implementation, and we could easily end up with some pages being encrypted under the old key, and some under the new key. That's a red flag.
  2. We'd have to invest quite some time in modifying SQLite3MultipleCiphers core code, as opposed to reusing interfaces that are already battle-tested.
  3. This whole ordeal is going to include VACUUM-ing the database and rewriting the WAL pages, so it will all happen under a huge write lock. That's not far from downtime from users' perspective anyway. And the space amplification has the same order of magnitude, because the dominant operation is VACUUM.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Begin by locating the libsql-server encryption implementation and the C API boundary for sqlite3_rekey, then review the SQLite3MultipleCiphers rekey behavior described in the issue. Done means safely stopping requests, checkpointing and switching journal modes around rekeying, restoring WAL mode, and validating that data is encrypted with the new key.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, sqlite
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.