Suggesting one consumer each runtime and self-management offset mode
- Dominant language
- Java
- Stars
- 1.8k
- Forks
- 658
- Avg merge
- 11h 29m
- Merged PRs (30d)
- 52
Description
## Enhancement Request
### Is your enhancement request related to a problem? Please describe
EventMesh currently chooses a consumer group-level agent, that is, there is only one real Consumer on a single EventMesh instance of a consumer group.
The number of clients that a single EventMesh instance can support is limited by the number of consumers and the average number of clients in every consumer group.
Each consumer has a private resource pool. In general, this resource pool includes but is not limited to a collection of threads, such as threads used to obtain messages and consumption, and some timing threads used to access the registry. When there are a limited number of consumers After being created on the EventMesh instance, a resource explosion will occur.
The producer has simple logic, takes up less resources and can be ignored.
like this, when several consumers are created, the JVM will not have enough threads, and the memory resources will run stably

Scene P1 and Scene P2 show the number of clients that a single EventMesh instance can support in two cases (on average, a single consumer group has more clients and fewer clients).

Obviously, in terms of quantity, scenario 1 is better In scene 2.
### Describe the solution you'd like
Global consumer, a single EventMesh instance has one and only one singleton Consumer & Producer, this singleton Consumer&Producer is responsible for all clients connected to the EventMesh to send and receive events, if a single EventMesh instance only has one global consumer,
include two parts
1. manage meta-data about client-group mapping
~ meta-data table like example
~ sync meta-data with broadcast

2. local ack manage
~ The consumption progress of EventMesh is managed separately from the consumption progress of the actual consumer group

### Describe alternatives you've considered
Contributor guide
Research direction
Start by reading the EventMesh consumer and producer runtime behavior described in the issue, especially consumer-group ownership, client-group metadata, broadcast synchronization, and local acknowledgement management. Define the singleton consumer/producer model and its metadata and acknowledgement requirements first; done means one runtime can serve all connected clients without per-consumer resource explosion while preserving consumption progress.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100