apache / apache/rocketmq

[Enhancement] batch send support compression 批量发送支持压缩

Open
#8,755 2 comments 0 reactions 0 assignees View on GitHub
type/enhancement
Dominant language
Java
Stars
22.6k
Forks
12k
Avg merge
2d 20h
Merged PRs (30d)
26

Description

### Before Creating the Enhancement Request

- [X] I have confirmed that this should be classified as an enhancement rather than a bug/feature.

### Summary

by default, `DefaultMQProducer` will compress message body when body length is longer than 4k, but if send with batch(`send( Collection msgs)`), message body are not compressed.
it's better compress each message body when average length of messages body in the batch is larger than 4k.

默认配置下,用`DefaultMQProducer`的send方法发送单条消息,消息体大于4MB会被压缩。但该压缩不支持批量消息,即用`send( Collection msgs)`发送的消息,造成io和磁盘空间的浪费。
希望批量发送接口也能支持压缩

### Motivation

trade cpu over disk io and space
使用cpu换取io和磁盘空间,一般的文本消息可以节约成本和提高效率。

### Describe the Solution You'd Like

compress each message body when average length of messages body in the batch is larger than 4k.
批量的消息大时可以压缩消息,与单条类似的机制

### Describe Alternatives You've Considered

1. 使用方自行压缩消息体
2. 批量消息一起压缩,包括header和body

### Additional Context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start at DefaultMQProducer.send(Collection msgs) and compare its handling with the single-message send path, especially the existing 4k compression behavior. Trace the batch message construction and its tests, then verify that batches whose average body length exceeds 4k compress each message while preserving normal send behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design, distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.