ydb-platform / ydb-platform/ydb-python-sdk

bug: Transactional topic example exhausts producer_id limit with random IDs

Open
#700 1 comment 0 reactions 1 assignee View on GitHub

@vgvoleg is already working on this.

Since Sep 17, 2026.

ai_reviewed bug
Dominant language
Python
Stars
102
Forks
75
Avg merge
1d 23h
Merged PRs (30d)
13

Description

Bug Report

YDB Python SDK version:

Environment

Environment is irrelevant for reproduction - issue is in code example

Current behavior:

In the transactional topic write example (topic_transactions_async_example.py):

  1. Each call to tx_writer without explicit producer_id generates a new random producer_id
  2. Transaction retries create additional producer_ids
  3. This rapidly consumes the producer_id limit (100K per 14 days)
  4. The example encourages production-unsafe usage patterns

Expected behavior:

Examples should demonstrate best practices:

  1. Explicit producer_id should be used to avoid random ID generation
  2. Example should be resilient to transaction retries
  3. Should show either fixed producer_id or partition_id usage
  4. Should not risk exhausting producer_id limits in production scenarios

Steps to reproduce:

  1. Use example from topic_transactions_async_example.py
  2. Run write_with_tx_example method in a loop or with frequent retries
  3. Observe rapid growth of producer_ids in the topic

Related code:
https://github.com/ydb-platform/ydb-python-sdk/blob/main/examples/topic/topic_transactions_async_example.py#L28

async def write_with_tx_example(driver: ydb.aio.Driver, topic: str, message_count: int = 10):
    async with ydb.aio.QuerySessionPool(driver) as session_pool:

        async def callee(tx: ydb.aio.QueryTxContext):
            tx_writer: ydb.TopicTxWriterAsyncIO = driver.topic_client.tx_writer(tx, topic)

            for i in range(message_count):
                async with await tx.execute(query=f"select {i} as res;") as result_stream:
                    async for result_set in result_stream:
                        message = str(result_set.rows[0]["res"])
                        await tx_writer.write(ydb.TopicWriterMessage(message))
                        print(f"Message {result_set.rows[0]['res']} was written with tx.")

        await session_pool.retry_tx_async(callee)

Other information:

Current example can exhaust 100K producer_id limit under moderate load

Possible Solutions
Fixed producer_id
Explicit partition selection

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.