lance-format / lance-format/lance-namespace-impls
feat: add Tencent GooseFS Table Master namespace implementation (Python)
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 11
- Forks
- 13
- PR merge metrics
- No merged PRs in 30d
Description
Summary
Add a new Lance Namespace implementation backed by Tencent Cloud GooseFS Table Master to the Python lance-namespace-impls package.
GooseFS Table Master is a Lance-native metadata service exposed over gRPC. Unlike general-purpose catalogs (Hive, Glue, Iceberg, Polaris, Unity), it is purpose-built for Lance — every registered table is a Lance table, so no table_type=lance marker filtering is required on the client side.
The new implementation acts as a thin pass-through layer that translates each Lance Namespace request into the corresponding *PRequest defined in the GooseFS Table Master gRPC schema and forwards it through the official goosefs-metastore-client library.
Motivation
- Provide first-class Lance Namespace support for users running Lance on top of Tencent Cloud GooseFS / COS.
- Cover the full Lance Namespace surface (basic CRUD + advanced operations such as indexing, tags, versioning, and transactions) without the overhead of mapping into a generic catalog model.
- Match the integration pattern already established for Hive, Glue, Iceberg, Polaris, and Unity.
Scope
Core implementation
- New
GooseFSNamespaceclass registered as"goosefs"inLanceNamespaces - 2-level namespace hierarchy (
database>table); identifier is forwarded verbatim to the Table Master - Configurable connection (
uri/host/port/timeout/max_retries), authentication (authentication_enabled/username/impersonation_user), and namespace-level defaults (manifest_enabled/dir_listing_enabled)
Operations covered
- Namespace: Create / List / Describe / Drop / Exists
- Table: Create / CreateEmpty / Declare / Register / List / Describe / Drop / Deregister / Exists / Rename
- Data plane: Insert / MergeInsert / Delete / Update / Query / CountRows
- Schema evolution: AddColumns / AlterColumns / DropColumns / UpdateSchemaMetadata
- Indexing: CreateIndex / CreateScalarIndex / ListIndices / DescribeIndexStats / DropIndex
- Tags & versioning: Create/Update/Delete/List/Get tag, Create/List/Describe version, Restore, plus Batch variants
- Transactions & query planning: AlterTransaction / DescribeTransaction / ExplainQueryPlan / AnalyzeQueryPlan / GetTableStats
Packaging & build
- New optional extra
goosefsinpython/pyproject.toml(goosefs-metastore-client==0.1.7,grpcio>=1.78,grpcio-status>=1.78) - New Makefile targets:
lint-goosefs,install-goosefs,test-goosefs,integ-test-goosefs - Unit tests:
python/tests/test_goosefs.py - Integration tests:
python/tests/test_goosefs_integration.py
Documentation
- New page
docs/src/goosefs.mddescribing background, configuration properties, object mapping, Lance-table identification, and per-operation semantics docs/src/.pagesupdated to includeTencent GooseFS: goosefs.mdin the navigation
Out of Scope
- A Java implementation — this PR only adds the Python binding. A Java counterpart can be added in a follow-up if there is demand.
- Bundling the GooseFS gRPC
.protofiles in this repo; they continue to ship insidegoosefs-metastore-client.
Verification
ruff format --checkandruff checkpass on the new files and across the wholepython/treepython -m compileallsucceeds forsrc/andtests/- Unit tests in
tests/test_goosefs.pypass with thegoosefsextra installed - Integration tests in
tests/test_goosefs_integration.pypass against a live GooseFS Table Master (skipped automatically when the service is not reachable)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Compare the existing Hive, Glue, Iceberg, Polaris, and Unity implementations before working in the Python lance-namespace-impls package. Review python/pyproject.toml, python/tests/test_goosefs.py, python/tests/test_goosefs_integration.py, docs/src/goosefs.md, and docs/src/.pages; done means the GooseFS registration, operations, packaging, documentation, linting, unit tests, and integration checks are complete.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- grpc, python
- Domain
- api, backend, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100