apache / apache/eventmesh

Cannot Push Kafka → EventMesh → HTTP Webhook (CloudEvents) despite correct config

Open
#5,212 2 comments 0 reactions 0 assignees View on GitHub
question Stale
Dominant language
Java
Stars
1.8k
Forks
658
Avg merge
11h 29m
Merged PRs (30d)
52

Description

### Search before asking

- [x] I had searched in the [issues](https://github.com/apache/eventmesh/issues?q=is%3Aissue) and found no similar issues.

### Question

Hi team,

We are trying to use EventMesh as a Kafka → HTTP push gateway using CloudEvents in EKS and are stuck even after coup days of troubleshooting.

Kafka (Confluent Platform – KRaft mode)

EventMesh Runtime (storage-plugin: kafka, protocol: cloudevents)

HTTP Consumer (Python server expecting POST events)

What works

EventMesh HTTP publish API:

POST /eventmesh/publish/demo-user
→ HTTP consumer receives the event correctly.

EventMesh logs repeatedly show:

Value Deserializers with error: io.cloudevents.kafka.CloudEventDeserializer

Even after switching to:

eventMesh.protocol.type=cloudevents
eventMesh.server.kafka.deserializer.type=cloudevents
eventMesh.server.kafka.value.deserializer=io.cloudevents.kafka.CloudEventDeserializer

EventMesh logs repeatedly show:

Value Deserializers with error: io.cloudevents.kafka.CloudEventDeserializer

Even after switching to:

eventMesh.protocol.type=cloudevents
eventMesh.server.kafka.deserializer.type=cloudevents
eventMesh.server.kafka.value.deserializer=io.cloudevents.kafka.CloudEventDeserializer

We need help determining:

Is the CloudEventDeserializer actually supported in runtime with Kafka storage?
Is binary-mode CloudEvents supported for Kafka → HTTP push?

Any working example of:
Kafka producer (CloudEvents)
EventMesh CloudEvents protocol
Kafka → EventMesh → Webhook push config

I can share logs, configs, YAML, and producer code if needed.

Thanks for your support

Contributor guide

Open the contributing guide

Research direction

Start with the EventMesh Runtime Kafka storage-plugin and CloudEvents protocol configuration shown in the issue, then inspect the logged CloudEventDeserializer failure and compare it with the HTTP publish path. Reproduce the Kafka → EventMesh → HTTP webhook flow with the stated Confluent KRaft setup; done means establishing whether the deserializer and binary-mode path are supported and documenting a working producer/configuration example.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, java, kafka, kubernetes
Domain
api, backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.