apache / apache/arrow-rs

Use a "SecureString" like type to store Parquet encryption keys

Open
#7,373 5 comments 0 reactions 0 assignees View on GitHub
enhancement parquet
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 16h
Merged PRs (30d)
168

Description

#6637 and #7111 added support for reading and writing Parquet files with encryption. These add `FileDecryptionProperties` and `FileEncryptionProperties` types that hold encryption keys as a `Vec`. Precaution should be taken to prevent accidentally exposing these keys and allowing unauthorised access to encrypted data.

In the C++ Parquet implementation for example, these keys are "wiped" after a file is read or written, which is intended to prevent any memory access bugs from being able to expose these keys. But it's known that this wiping isn't very secure as only the first byte of the key is usually overwritten. See https://github.com/apache/arrow/issues/31603 and some of the discussion in https://github.com/apache/arrow/pull/44990.

Ideally these keys should be stored in a type that automatically clears the whole key from memory when it is dropped, eg. something like https://crates.io/crates/secure-string, or a custom abstraction built on top of https://crates.io/crates/zeroize.

We might also want to have a `Debug` implementation that doesn't show the key contents to avoid accidental logging of keys.

Contributor guide

Open the contributing guide

Research direction

Start by locating the FileDecryptionProperties and FileEncryptionProperties definitions and reviewing how their key Vec values are stored, cleared, and formatted for Debug. Evaluate the stated secure-string or zeroize approach; done means the full key is cleared on drop and key contents are not exposed by Debug.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cryptography, security
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.