InditexTech / InditexTech/scs-outbox

Evaluate batching outbox message persistence during transaction capture

Open
#92 0 comments 0 reactions 0 assignees View on GitHub
kind/feature
Dominant language
Java
Stars
19
Forks
0
Avg merge
9h 31m
Merged PRs (30d)
17

Description

### Motivation

A single business transaction can produce multiple outbound events. At the moment, each captured event is persisted immediately through an individual repository operation.

This means that a transaction producing many events performs one database operation per event during the outbox capture phase. The individual operations participate in the same transaction, but the repeated database round trips can increase the time required to complete the business transaction unnecessarily.

The library should assess whether captured outbox messages can be accumulated during the current transaction and persisted using a batch operation, such as a JDBC batch insert or a MongoDB multi-document insert. This could reduce the number of database round trips and improve capture performance for transactions that produce many events.

### Acceptance criteria

- [ ] Measure the current capture performance and database operation count for transactions producing multiple outbox messages.
- [ ] Assess the feasibility of accumulating messages per transaction and persisting them using a batch operation.
- [ ] Evaluate the required changes to the `OutboxMessageRepository` contract and the capture flow.
- [ ] Evaluate a JDBC batch implementation for the supported databases, including PostgreSQL and MariaDB.
- [ ] Evaluate a MongoDB multi-document insert implementation.
- [ ] Preserve the atomicity of the business transaction and ensure that all captured messages are rolled back when the transaction rolls back.
- [ ] Preserve the order in which messages are captured.
- [ ] Define the behavior when the transaction contains only one captured message.
- [ ] Define the behavior for serialization errors, database errors, and partial batch failures.
- [ ] Define whether a configurable maximum batch size is required to control memory usage and database limits.
- [ ] Assess the impact on capture metrics, logging, and observability.
- [ ] Compare the performance, complexity, and operational trade-offs against the current individual persistence approach.
- [ ] Add unit and integration tests for multiple messages in one transaction, rollback, failures, ordering, JDBC, and MongoDB.
- [ ] Document the resulting behavior and any limitations if batching is implemented.

Contributor guide

Open the contributing guide

Research direction

Start with the OutboxMessageRepository contract and the transaction capture flow, then measure current operation counts for transactions producing multiple messages. Compare JDBC batching for PostgreSQL and MariaDB with MongoDB multi-document inserts, and document the chosen behavior, transactional guarantees, ordering, failures, observability, and required unit and integration tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mariadb, mongodb, postgresql
Domain
backend, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.