[PEP-Request] ThriftMessageDecoder implementation for thrift stream ingestion
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 55m
- Merged PRs (30d)
- 182
Description
_I tested this feature in local, and want to contribute to this project._
**1. What needs to be done?**
: Implement "ThriftMessageDecoder" for Thrift Stream Ingestion
**2. Why the feature is needed (e.g. describing the use case).**
: Here is example case
- I work on AI team and our services communicate based on thrift.
- We stack log data on kafka.
- I want to stream ingest thrift data through kafka, and add OLAP dashboards.
**3. It may also include an initial idea/proposal on how as well.**
- locate thrift-jar file to /lib folder
- using ThriftRecordExtractor & complexTypeConfig for decoding
Contributor guide
Research direction
Locate the Thrift stream-ingestion entry points and inspect how ThriftRecordExtractor and complexTypeConfig are used for decoding. Check the proposed thrift JAR placement under /lib and existing Kafka ingestion tests; done means ThriftMessageDecoder can decode the described Thrift records for stream ingestion and the relevant tests pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, kafka
- Domain
- data-engineering, stream-processing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100