lance-format / lance-format/lance

bug: OSS branch listing omits names containing slashes

Open
#9,197 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug performance
Dominant language
Rust
Stars
7.1k
Forks
852
Avg merge
3d 18h
Merged PRs (30d)
272

Description

Description

On OpenDAL-backed object stores such as Aliyun OSS, a branch whose logical name contains / can be checked out by its full name but is omitted from list_branches() / ds.branches.list().

For example, after creating features/example, the branch metadata can be read by an exact lookup and checkout_version(("features/example", version)) succeeds, while branch listing does not include features/example.

Root cause

Branches::fetch uses the non-recursive ObjectStore::read_dir on _refs/branches. The branch metadata path is constructed with / encoded as %2F, but some OpenDAL adapters decode that separator when translating the path into a provider key. OSS therefore exposes the metadata during LIST as:

_refs/branches/features/example.json

The non-recursive listing returns only the features common prefix. Since it does not end in .json, Lance filters it out. Exact lookup still works because the same path translation is applied to GET.

Expected behavior

Branch listing should return the complete logical branch name on every supported object store, independent of whether the adapter exposes encoded separators as a flat key or nested path.

Impact

This affects more than discovery. Internal operations that enumerate branches, including dependency checks and cleanup, can also receive an incomplete branch set.

Environment
  • Lance main at 49223d319
  • Python binding
  • Aliyun OSS through OpenDAL

I have a focused fix and regression test that simulates both the flat and nested listing layouts.

Related work

#8403 proposed a much broader, unmerged redesign for #7185 that would also decouple logical branch names from physical storage paths. The current main branch still uses name-backed metadata and remains affected by this focused listing bug.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at Branches::fetch and its non-recursive ObjectStore::read_dir call for _refs/branches. Reproduce listing against both flat and nested layouts, then verify that branch names containing slashes are returned completely and that the regression coverage passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.