lance-format / lance-format/lance-namespace-impls
Catalog impls access request fields as attributes but pylance's Rust bridge passes a DictWithModelDump (AttributeError on describe_table write path)
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 11
- Forks
- 13
- PR merge metrics
- No merged PRs in 30d
Description
Summary
When pylance writes a Lance table to a namespace-backed catalog (e.g. Glue) and its Rust storage-options provider calls back into the Python namespace impl's describe_table, it passes the request as a DictWithModelDump (a plain dict subclass exposing only model_dump(), no attributes). But GlueNamespace.describe_table (and the other catalog impls) access the request via attribute (request.load_detailed_metadata), which raises:
AttributeError: 'DictWithModelDump' object has no attribute 'load_detailed_metadata'
This breaks the write path (lance.dataset(...).write / lance_ray.write_lance) for every catalog impl in this package when driven through pylance's Rust namespace bridge.
Environment (a concrete, reproducing set)
pylance == 9.0.0(pinslance-namespace>=0.8.5,<0.9)lance-namespace == 0.8.6lance-namespace-urllib3-client == 0.8.6lance-namespace-impls == 0.4.1lance-ray == 0.5.0
Root cause
pylance's PyO3 bridge (python/src/namespace.rs, present across 6.x–10.x) builds a request, serializes it to JSON, wraps it in a DictWithModelDump, and invokes the Python namespace impl's method with that dict. pylance's own wrappers (DirectoryNamespace/RestNamespace in lance/namespace.py) tolerate this because they call request.model_dump(). But the impls in this package access request fields as attributes.
describe_table in lance_namespace_impls/glue.py (v0.4.1, lines 353–358):
def describe_table(self, request: DescribeTableRequest) -> DescribeTableResponse:
"""Describe a table."""
if request.load_detailed_metadata: # <-- AttributeError on DictWithModelDump
raise RuntimeError(
"load_detailed_metadata=true is not supported for this implementation"
)
The same attribute-access pattern (request.<field>) appears in all catalog impls: glue.py, hive2.py, hive3.py, unity.py, iceberg.py, polaris.py.
Reproduction (isolated)
from lance_namespace_impls.glue import GlueNamespace
class DictWithModelDump(dict):
def model_dump(self): return dict(self)
ns = GlueNamespace.__new__(GlueNamespace)
ns.describe_table(DictWithModelDump({"id": ["default", "t"], "load_detailed_metadata": True}))
# -> AttributeError: 'DictWithModelDump' object has no attribute 'load_detailed_metadata'
End-to-end, this surfaces during a Ray → Glue write as:
OSError: LanceError(IO): Failed to fetch storage options: ... Python error in describe_table:
AttributeError: 'DictWithModelDump' object has no attribute 'load_detailed_metadata',
src/namespace.rs:1571 ; lance-io/src/object_store/storage_options.rs
Suggested fix
Make the impls accept a dict-style request (the shape pylance's Rust bridge actually passes), e.g. coerce at the top of each request-taking method:
from lance_namespace_urllib3_client.models import DescribeTableRequest
def describe_table(self, request) -> DescribeTableResponse:
if not isinstance(request, DescribeTableRequest):
request = DescribeTableRequest.from_dict(dict(request))
...
(Verified working as a monkeypatch shim against the environment above — coercing DictWithModelDump → the real Pydantic model makes attribute access succeed and describe_table returns the correct storage_options.) Alternatively, pylance's call_py_method could pass a real model instead of DictWithModelDump.
The other five impls (hive2, hive3, unity, iceberg, polaris) need the same treatment for their request-taking methods.
Interim workaround
A startup shim (installed via a .pth so it runs in every process) that wraps the impl methods and coerces dict → model before delegating. Happy to open a PR with the in-method coercion if that direction is preferred.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the request-taking methods in glue.py, hive2.py, hive3.py, unity.py, iceberg.py, and polaris.py, then reproduce the isolated DictWithModelDump failure described in the issue. Compare these methods with the wrappers in lance/namespace.py and verify that each catalog implementation handles the Rust bridge request and completes the describe_table write path without AttributeError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100