apple / apple/foundationdb

API for string prefix queries within Tuples

Open
#282 0 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

Sometimes, there is a need to do string prefix queries inside a tuple element (i.e. all tuples (a, b, str…, …) that start with (a, b, str)). Our current range query does not provide the ability to treat str as a prefix, rather it returns all sub-tuples with str as the last element in the prefix tuple, i.e. (a, b, str, …).

It’s actually somewhat complicated for people to implement this themselves. There are multiple choices for begin key that would determine which of the following keys would be the first included:

1. (a, b, str)
2. (a, b, str, …)
3. (a, b, str…)
4. (a, b, str…, …)

There is no key that separates 3 and 4 in general, but you could draw a line before 1, 2, or 3.

Then there’s the matter of the end key. I believe the strictly correct end key is the encoding of (a, b, sts) with the trailing null byte removed (sts is obtained by running strinc(str)), but we don’t publish strinc in all of our bindings, and even then it only works on byte arrays. Also, it fails if called on a string consisting only of 0 or more \xff characters. It can be used on \xff strings or unicode by packing (a, b, str), removing the trailing null byte, and then passing the result to strinc, but that trick is dependent on our particular encoding, and it’s not at all obvious that it should work.

An API for this could produce the correct end key as well as 1 or more options for the begin key without requiring the user to reason through all of this and risk getting it wrong in subtle ways.

This should work on all Tuple types that one could want to do prefix queries on, the obvious ones being byte strings and Unicode strings. It's possible that this could also apply to nested Tuples.

Contributor guide

Open the contributing guide

Research direction

Start by locating the existing range-query and Tuple APIs across the bindings, then review how strinc and string or byte-string encoding are exposed. Define the supported prefix cases and verify begin and end keys for byte strings, Unicode strings, and any nested Tuples. Done means callers can perform these prefix queries without reproducing encoding logic or missing the stated edge cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend-api-design, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.