Basekick-Labs / Basekick-Labs/arc

Research: SparkplugB ingestion support

Open
#277 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Go
Stars
677
Forks
53
Avg merge
9h 14m
Merged PRs (30d)
164

Description

## Context

SparkplugB is a specification built on top of MQTT that defines a standardized topic namespace, payload encoding (Protocol Buffers), and state management for industrial IoT (IIoT) and SCADA systems. It is widely used in manufacturing, energy, and oil & gas for device-to-cloud telemetry.

Arc already has native MQTT integration — adding SparkplugB support would allow direct ingestion of industrial telemetry without requiring middleware to decode Protobuf payloads into JSON or MessagePack.

## Research Goals

1. **Market demand** — Identify users, companies, or communities interested in SparkplugB ingestion. Check:
- Eclipse Sparkplug working group and adopters
- Industrial IoT forums (HiveMQ community, Cirrus Link, Inductive Automation/Ignition users)
- GitHub issues/discussions on similar projects (QuestDB, TimescaleDB, InfluxDB, CrateDB) requesting SparkplugB
- Reddit r/PLC, r/SCADA, r/IoT threads

2. **Protocol analysis** — Understand the SparkplugB payload format:
- Protobuf schema (`sparkplug_b.proto`) — metrics, datatypes, birth/death certificates
- Topic namespace: `spBv1.0/{group_id}/{message_type}/{edge_node_id}/{device_id}`
- Message types: NBIRTH, NDEATH, DBIRTH, DDEATH, NDATA, DDATA, NCMD, DCMD
- Metric types: Int8-Int64, UInt8-UInt64, Float, Double, Boolean, String, DateTime, **Template** (nested)
- State management: birth certificates define the metric schema, data messages reference metrics by alias (uint64) for bandwidth efficiency

3. **Integration approach** — Evaluate implementation options:
- Option A: Decode SparkplugB Protobuf in the existing MQTT subscriber (add a `format: sparkplugb` subscription option)
- Option B: Dedicated `/api/v1/write/sparkplugb` endpoint for users who handle MQTT externally
- Option C: Both — MQTT native + HTTP endpoint
- Consider: metric alias resolution, birth certificate caching, template/nested metric flattening, tag extraction from topic namespace

4. **Competitive landscape** — How do other databases handle SparkplugB?
- HiveMQ + InfluxDB/TimescaleDB pipelines
- Ignition + database connectors
- Any native SparkplugB support in analytical databases?

## Output

Summary of findings with a recommendation on whether to proceed and which integration approach to take.

---

👍 **If you'd like to see SparkplugB ingestion support in Arc, please react to this issue with a thumbs up!** This helps us prioritize based on community interest.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.