meilisearch / meilisearch/heed

Is RwTxn Send?

Open
#339 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
914
Forks
82
PR merge metrics
No merged PRs in 30d

Description

I found that multi-threaded tokio runtime moves the task between threads, and this leads to let tx = env.write_txn() and tx.commit() to run in different threads. And the next call to env.write_txn() hangs.

let tx = env.write_txn()?; // thread A

while let Some(row) = stream.next().await {
    insert_row(&mut tx, row)?;
}

tx.commit()?;  // thread B

let tx = env.write_txn()?;  // hangs
[indexing] IndexBuilder tx is created in thread=ThreadId(9)
████████████████████████████████████████ 4317834/4317834
[indexing] IndexBuilder tx is committed in thread=ThreadId(17)
[task] lmdb - getting WriteTx in thread=ThreadId(17)  # hangs

Switching to a single-threaded tokio runtime did not have a deadlock.

I found this in https://github.com/jnwatson/py-lmdb/issues/89#issuecomment-109167232 :

A write Transaction can only be used from the thread it was created on.

Is it okay to commit a write transaction in a different thread?
If not, what is a recommended approach to interleave async workload with transactions?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the multi-threaded Tokio reproducer shown in the issue and trace the write_txn and commit calls across the reported thread changes. Check the transaction API and LMDB thread-safety expectations, then determine the supported approach for interleaving async work with transactions. Done means the cross-thread commit behavior and a recommended usage pattern are clearly established.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.