MetaCell / MetaCell/cloud-harness

Extract common Kafka stream methods from EventClient

Open
#355 5 comments 0 reactions 0 assignees View on GitHub
scope:events scope:python-runtime
Dominant language
Python
Stars
19
Forks
5
Avg merge
3d 18h
Merged PRs (30d)
3

Description

~~It seems from the kafka-python api [docs](https://kafka-python.readthedocs.io/en/master/apidoc/KafkaConsumer.html) that a KafkaConsumer can receive an optional list of topics to subscribe to. It would be nice that [EventClient](https://github.com/MetaCell/cloud-harness/blob/develop/libraries/cloudharness-common/cloudharness/events/client.py#L48) would allow the same instead of forcing one and one only topic.~~

Edited after discussing with @zsinnema:

CH could have the common Kafka stream methods extracted to a new class, so that it can support different use cases

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in libraries/cloudharness-common/cloudharness/events/client.py at the EventClient implementation, then compare its behavior with the kafka-python KafkaConsumer API. Identify the common Kafka stream methods that belong in a new class; the work is done when EventClient and other intended use cases can use that shared abstraction, including the discussed topic-subscription behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
kafka, python
Domain
stream-processing
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.