lance-format / lance-format/lance

Add tag creation and update timestamps to the Lance format

Open
#6,333 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Rust
Stars
7.1k
Forks
852
Avg merge
3d 18h
Merged PRs (30d)
272

Description

Currently, tag metadata in the Lance format is enough to resolve a tag to a target version or branch, but it does not expose lifecycle timestamps. For higher-level management services, we want the Lance format itself to provide both tag creation
time and last update time, so the dataset can serve as the source of truth for tag metadata.

This is useful for management workflows such as:

  • showing tag creation and last modification time in APIs and UIs
  • sorting and filtering tags by creation time or recency
  • supporting audit and troubleshooting workflows
  • avoiding the need for an external metadata store just to track tag timestamps

We should extend tag metadata in the Lance format to include:

  • createdAt: the time when the tag was first created
  • updatedAt: the time when the tag was last updated

Expected behavior:

  • when a tag is created, both createdAt and updatedAt should be set
  • when a tag is updated to point to a different reference, createdAt should remain unchanged and updatedAt should be refreshed
  • older datasets without these fields should remain readable for backward compatibility
  • for legacy tags that do not have createdAt, the field may remain absent rather than being backfilled with an inferred value

This metadata should be surfaced consistently to upper-layer integrations and management services so they can consume tag timestamps directly from Lance without reconstructing them from external state.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No implementation files or tests are named. Start by locating the Lance tag metadata definition and its serialization and deserialization paths, then inspect existing tag creation and update handling. Done means timestamps are exposed with the required create/update semantics, older datasets remain readable, and relevant format or integration tests cover legacy tags.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.