meilisearch / meilisearch/heed

Checking for already opened LMDB Environments

Open
#180 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Rust
Stars
914
Forks
82
PR merge metrics
No merged PRs in 30d

Description

Heed strive to pursue opening LMDB safely, so a database environment must be opened only once inside a process address space.

Any subsequent opening shall return the corresponding already opened environment.

It was done with Path Canonization but it fails to tackle symlinks, hardlinks and renaming/move of the whole database environment (both locks and database).

In an ideal world the environment already open checking must prevail:

  • The solution should avoid keeping the environment "open" aka holding a file descriptor elsewhere see https://github.com/meilisearch/heed/pull/179#issue-1795327221
  • First opening
  • Opening with the same path
  • Access through symlink
  • [ ] Access through hardlink Not possible since hard_link only apply to files and we track dir for envs.
  • Renaming

Related pull requests/ Issue

Open question

Is Canonization still pertinant if we check already-openned by Unix: (device/inode) Windows: (Driver/fileID) through same_file?

How to do it compatible with windows without lefting file open.

Problems

Lmdb rely on Unixes on posix filelocking through fnctl

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading related pull request #179, issue #145, and the LMDB assumptions in issue #20, then review the same_file Handle documentation. Define how already-open environments should be detected for repeated paths, symlinks, and renames on Unix and Windows without retaining an extra file descriptor; done means the checklist cases are covered and the open questions are resolved.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.