Standardize tuple encoding of malformed unicode strings
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
There is some varying behavior between our bindings with respect to how they handle malformed unicode strings. The Java bindings now treat these as errors (see https://github.com/apple/foundationdb/pull/1317), but Python does not (at least not universally), and in some cases it apparently does things that aren't great. For example, I believe that with some strings, you lose the property that `pack(unpack(tup))` == `tup` (or maybe it's the other order, I think @alecgrieser could provide specific details).
I'm not sure any effort has been made to look at what other bindings do with these strings. I propose we define what the behavior for our tuple encoding should be for malformed unicode strings and then implement as best we can across each of our bindings.
Contributor guide
Research direction
Start by comparing how the Java and Python bindings handle malformed Unicode strings during tuple pack and unpack operations. Review the behavior described in pull request 1317 and determine the intended round-trip property before surveying the other bindings. Done means the behavior is defined and implemented consistently across each binding.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, python
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100