influxdata / influxdata/influxdb

data from confluent cloud to influxdb cloud

Open
#24,968 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

Steps to reproduce:
List the minimal actions needed to reproduce the behavior.

Configure Kafka to InfluxDB connector in Confluent Cloud with correct InfluxDB credentials and data mappings.
Send data to the specified Kafka topic.
Check InfluxDB for the expected data.
Expected behaviour:
Data sent from Confluent Cloud via the Kafka-InfluxDB connector should appear in the specified bucket in InfluxDB Cloud.

Actual behaviour:
While the Confluent Cloud logs indicate that data is successfully passing through to InfluxDB, no corresponding data entries are visible in the InfluxDB Cloud bucket.

Environment info:

Command used to build the project: [Include here if applicable, such as a Docker command or a command-line invocation]
System info: [Output of uname -srm or equivalent system information command]
Containerised environment details (if applicable): [e.g., Docker version, Kubernetes environment]
Other relevant environment details: disk info, hardware setup, network info etc.
Config:
Copy of the Kafka to InfluxDB connector configuration (with sensitive details redacted).

Logs:
![image](https://github.com/influxdata/influxdb/assets/46783175/247334ce-1ad3-483d-8c8c-51826b9039ff)

Include relevant snippets from the Confluent Cloud connector logs showing successful data handling.
Include logs or error messages from InfluxDB if available.
If possible, add results from RUST_BACKTRACE=full and verbose logging from IOx server with -vv.
Additional Information:

Detail any specific settings or peculiarities in your Kafka to InfluxDB configuration that might affect data ingestion.
Mention any recent changes in the configuration or updates in the system.

Contributor guide

Open the contributing guide

Research direction

Start by collecting the Kafka-to-InfluxDB connector configuration, Confluent Cloud logs, and the InfluxDB Cloud bucket details, since the issue currently contains placeholders for these diagnostics. Reproduce the transfer and compare the connector's successful-handling logs with the bucket contents; done means identifying why the expected entries are absent or documenting the missing information needed to continue.

Written by the indexing model from the issue text.

Assessment

Tech stack
kafka
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.