apple / apple/foundationdb

Proposal: keys-only range reads for API 800 with C, Java, and CLI support

Open
#14,039 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

We'd like to add `getRangeKeys` so clients can scan a range without fetching its values. This follows #5789 and #7417, where people asked for the same capability and suggested filtering at the storage server. We'd like to target 8.0 / API 800 if the release timing allows it.

Today, applications fetch key/value pairs and discard the values. Even `fdbcli getrangekeys` does this. When values are much larger than keys, we spend bandwidth and allocate memory for data the caller never uses.

We propose adding:

- Native C++ `getRangeKeys(...)`, with the same selectors, limits, reverse reads, and snapshot semantics as `getRange(...)`.
- Two new C functions: `fdb_transaction_get_range_keys(...)` to start the read, and `fdb_future_get_range_keys(...)` to extract an `FDBKey` array, count, and `more` flag. Neither function exists upstream today.
- Java `ReadTransaction.getRangeKeys(...)` overloads returning `AsyncIterable`.

We'd also route the existing CLI command through the new operation.

The storage server would return only keys. We'd reuse its existing range traversal and MVCC merge, and push the filtering into memory, Redwood, RocksDB, and sharded RocksDB where practical. Other engines could start with a full-read projection fallback. Disk blocks may still contain values, so we expect fewer copies and smaller replies, but we need measurements before claiming less physical I/O.

### Compatibility

Applications would need updated headers/bindings and a library that exposes the new API. We propose these execution paths:

| Configuration | Execution path |
| --- | --- |
| Updated client, selected API 800+, compatible server | Use the keys-only storage-server operation for eligible reads |
| Updated client, selected API below 800 | Perform ordinary `getRange` and project keys inside the native client |
| Updated multiversion wrapper, external client missing the new functions | Use the same native projection fallback |
| Transactions with local mutations, or special-key reads | Initially use the existing range path and project keys |

We would preserve conflicts, pagination, cancellation, and result ownership. A value-only update would still conflict with a serializable key-range read.

The wire protocol needs a separate decision. API selection doesn't tell us whether a server supports the new RPC. We'd like to agree on a development-protocol bump or explicit capability negotiation before settling the 8.0 compatibility contract.

We have a local prototype covering the native client, C, Java, CLI, and the four engines. Focused integration and simulation checks pass. We'll validate the upstream port, run broader checks, and compare it with `getRange` followed by key projection across different value sizes.

### Open questions

- Would you prefer a dedicated `getRangeKeys` operation with a typed key result, or a projection option on `getRange`?
- Is API 800 the right introduction point? Should the updated native library also support this operation when applications select an older API?
- Should 8.0 use a development-protocol bump, or negotiate support and fall back for existing prerelease binaries?
- What additional semantics or validation should we cover before upstreaming?

Contributor guide

Open the contributing guide

Research direction

Start by reviewing prior issues #5789 and #7417, then trace the existing getRange paths and the named C, Java, and CLI entry points. Compare the local prototype with current upstream behavior and run the focused integration and simulation checks. Done requires an agreed operation and protocol or capability strategy, compatible fallbacks, preserved semantics, and broader validation against ordinary range reads.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, cpp, java
Domain
api, cli, databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.