influxdata / influxdata/telegraf

XPath Protobuf Parser incorrectly parses google.protobuf.Timestamp

Open
#17,115 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Go
Stars
17.8k
Forks
5.8k
Avg merge
1d 20h
Merged PRs (30d)
161

Description

### Relevant telegraf.conf

```toml
[agent]
hostname = "localhost"
flush_interval = "1s"

[[inputs.mqtt_consumer]]
servers = ["tcp://localhost:1883"]
client_id = "telegraf-test-message-consumer"
topics = ["test/message"]
data_format = "xpath_protobuf"
xpath_protobuf_file = "test.proto"
xpath_protobuf_type = "TestMessage"

[[inputs.mqtt_consumer.xpath]]
field_name_expansion = true
metric_name = "'TestMessage'"
field_selection = "timestamp"

[[processors.converter]]
namepass = ["TestMessage"]
[processors.converter.fields]
# timestamp = ["timestamp"]
# timestamp_format = "unix_ns"

[[outputs.file]]
files = ["stdout"]
data_format = "json"
```

### Logs from Telegraf

```text
{"fields":{"timestamp":"5690660001749138879"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138879} 👀
{"fields":{"timestamp":"1749138880574789000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138880}
{"fields":{"timestamp":"1749138881581062000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138881}
{"fields":{"timestamp":"1749138882587107000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138882}
{"fields":{"timestamp":"1749138883593769000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138883}
{"fields":{"timestamp":"1749138884600063000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138884}
{"fields":{"timestamp":"6083220001749138885"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138885} 👀
{"fields":{"timestamp":"1749138886616430000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138886}
{"fields":{"timestamp":"1749138887625780000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138887}
{"fields":{"timestamp":"1749138888632401000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138888}
{"fields":{"timestamp":"1749138889644017000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138889}
{"fields":{"timestamp":"1749138890654236000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138890}
{"fields":{"timestamp":"1749138891664063000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138891}
{"fields":{"timestamp":"1749138892677200000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138892}
{"fields":{"timestamp":"1749138893684626000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138893}
{"fields":{"timestamp":"1749138894695863000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138894}
{"fields":{"timestamp":"7017550001749138895"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138895} 👀
{"fields":{"timestamp":"1749138896715375000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138896}
{"fields":{"timestamp":"1749138906859505000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138906}
{"fields":{"timestamp":"1749138907867704000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138907}
{"fields":{"timestamp":"1749138908873501000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138908}
{"fields":{"timestamp":"1749138909879589000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138909}
{"fields":{"timestamp":"1749138910886459000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138910}
{"fields":{"timestamp":"8925670001749138911"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138911} 👀
{"fields":{"timestamp":"1749138912898451000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138912}
{"fields":{"timestamp":"1749138913904408000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138913}
{"fields":{"timestamp":"9107130001749138914"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138914} 👀
{"fields":{"timestamp":"1749138915917619000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138915}
{"fields":{"timestamp":"1749138916924037000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138916}
{"fields":{"timestamp":"9304070001749138917"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138917} 👀
{"fields":{"timestamp":"1749138918937308000"},"name":"stats","tags":{"topic":"test/message"},"timestamp":1749138918}
```

### System info

Telegraf 1.34.4, WSL: Ubuntu-24.04

### Docker

_No response_

### Steps to reproduce

Ref: https://protobuf.dev/reference/protobuf/google.protobuf/#timestamp

1. Generate proto message with current time in python and send as payload in MQTT
```
msg = TestMessage()
msg.timestamp.GetCurrentTime()
```
2. Start telegraf with config
3. Log data and notice some timestamps are rolled over e.g. "timestamp":"5690660001749138879"

### Expected behavior

e.g. "timestamp":"1749138880574789000"

### Actual behavior

"timestamp":"5690660001749138879" (sometimes because I assume the nano second affects this)

### Additional info

Proto file

```
edition = "2023";

import "google/protobuf/timestamp.proto";

message TestMessage {
google.protobuf.Timestamp timestamp = 1;
}
```

Contributor guide

Open the contributing guide

Research direction

Start with the mqtt_consumer entry point and its xpath_protobuf data format, using the supplied test.proto and Telegraf configuration to reproduce the incorrect google.protobuf.Timestamp values. Compare the parser output with the expected nanosecond timestamp; the work is done when valid Timestamp values remain unchanged across the reported nanosecond cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.