apple / apple/foundationdb

Support as_foundationdb_key interface objects in Python tuple layer

Open
#4,230 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

Currently, users can write

```python
obj = # instance of class with as_foundationdb_key() method
db[obj] = "test"
tr[obj] ="transaction_test"
```

However, when trying to use an object with an `as_foundationdb_key` method with the Subspace or Tuple layer, one gets `ValueError: Unsupported data type`. For example:

```python
fdb.tuple.pack(("some_space", 1, obj))

# or
subspace[obj] = "test_subpace"
```

Full error traceback

```
Traceback (most recent call last):
File "./test.py", line 20, in
g.add((donna, RDF.type, FOAF.Person))
File "/home/alex/miniconda/envs/tdb/lib/python3.7/site-packages/rdflib/graph.py", line 389, in add
self.__store.add((s, p, o), self, quoted=False)
File "/home/alex/c2/tdb/rdflib_foundationdb/store.py", line 168, in add
tr_add(self.db, triple, self.store)
File "/home/alex/miniconda/envs/tdb/lib/python3.7/site-packages/fdb/impl.py", line 266, in wrapper
ret = func(*largs, **kwargs)
File "/home/alex/c2/tdb/rdflib_foundationdb/store.py", line 79, in tr_add
xs = subspace[b"spo"][spo]
File "/home/alex/miniconda/envs/tdb/lib/python3.7/site-packages/fdb/subspace_impl.py", line 35, in __getitem__
return Subspace((name,), self.rawPrefix)
File "/home/alex/miniconda/envs/tdb/lib/python3.7/site-packages/fdb/subspace_impl.py", line 29, in __init__
self.rawPrefix = fdb.tuple.pack(prefixTuple, prefix=rawPrefix)
File "/home/alex/miniconda/envs/tdb/lib/python3.7/site-packages/fdb/tuple.py", line 401, in pack
res, version_pos = _pack_maybe_with_versionstamp(t, prefix)
File "/home/alex/miniconda/envs/tdb/lib/python3.7/site-packages/fdb/tuple.py", line 385, in _pack_maybe_with_versionstamp
child_bytes, version_pos = _reduce_children(map(_encode, t))
File "/home/alex/miniconda/envs/tdb/lib/python3.7/site-packages/fdb/tuple.py", line 291, in _reduce_children
for child_bytes, child_pos in child_values:
File "/home/alex/miniconda/envs/tdb/lib/python3.7/site-packages/fdb/tuple.py", line 366, in _encode
child_bytes, version_pos = _reduce_children(map(lambda x: _encode(x, True), value))
File "/home/alex/miniconda/envs/tdb/lib/python3.7/site-packages/fdb/tuple.py", line 291, in _reduce_children
for child_bytes, child_pos in child_values:
File "/home/alex/miniconda/envs/tdb/lib/python3.7/site-packages/fdb/tuple.py", line 366, in
child_bytes, version_pos = _reduce_children(map(lambda x: _encode(x, True), value))
File "/home/alex/miniconda/envs/tdb/lib/python3.7/site-packages/fdb/tuple.py", line 370, in _encode
raise ValueError("Unsupported data type: " + str(type(value)))
ValueError: Unsupported data type:
```

To improve the consistency and ease of use of the API, I propose
- checking objects for the `as_foundationdb_key()` method
- if present, calling the method
- then storing the result as bytes in the tuple layer

I'd love any thoughts or feedback. I'm open to writing a PR if it's welcome.

Thanks for the project!

Contributor guide

Open the contributing guide

Research direction

Start in fdb/tuple.py at the _encode path used by fdb.tuple.pack, and trace how values are converted before the Unsupported data type error. Verify the behavior with the shown tuple.pack and Subspace examples; done means objects exposing as_foundationdb_key() are converted to bytes and accepted by the tuple layer.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
databases
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.