influxdata / influxdata/telegraf

xpath fails to parse json array elements

Open
#18,145 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Go
Stars
17.8k
Forks
5.8k
Avg merge
1d 20h
Merged PRs (30d)
161

Description

### Relevant telegraf.conf

```toml
[[inputs.file]]
files = ["/etc/telegraf/smartctl-telemetry/nvme*.json"]
data_format = "xpath_json"

# Measurement name override (table name)
name_override = "system-nvme"
interval = "5s"

[[inputs.file.xpath]]
[inputs.file.xpath.tags]
device = "string(/device/name)"
model_name = "string(/model_name)"
serial_number = "string(/serial_number)"
[inputs.file.xpath.fields_int]
# Disk info (could also be tags, but in case the device changes,
# this could change too, so better keep as values).
logical_block_size = "number(/logical_block_size)"
id = "number(//nvme_namespaces[1]/id)"
capacity = "number(//nvme_namespaces[1]/capacity/bytes)"
utilization = "number(//nvme_namespaces[1]/utilization/bytes)"
# Wear data
available_spare = "number(/nvme_smart_health_information_log/available_spare)"
percentage_used = "number(/nvme_smart_health_information_log/percentage_used)"
data_units_read = "number(/nvme_smart_health_information_log/data_units_read)"
data_units_written = "number(/nvme_smart_health_information_log/data_units_written)"
# Power data
power_cycles = "number(/nvme_smart_health_information_log/power_cycles)"
power_on_hours = "number(/nvme_smart_health_information_log/power_on_hours)"
unsafe_shutdowns = "number(/nvme_smart_health_information_log/unsafe_shutdowns)"
# Errors
media_errors = "number(/nvme_smart_health_information_log/media_errors)"
num_err_log_entries = "number(/nvme_smart_health_information_log/num_err_log_entries)"
# Controller
controller_busy_time = "number(/nvme_smart_health_information_log/controller_busy_time)"
# Temperature
temperature = "number(/temperature/current)"
warning_temp_time = "number(/nvme_smart_health_information_log/warning_temp_time)"
critical_comp_time = "number(/nvme_smart_health_information_log/critical_comp_time)"
```

### Logs from Telegraf

```text
# telegraf --config /etc/telegraf/telegraf.conf --debug --test
2025-12-22T19:36:25Z W! Strict environment variable handling will be the new default starting with v1.38.0! If your configuration works with strict handling or you don't use environment variables it is safe to ignore this warning. Otherwise please explicitly add the --non-strict-env-handling flag!
2025-12-22T19:36:25Z I! Loading config: /etc/telegraf/telegraf.conf
2025-12-22T19:36:25Z I! Starting Telegraf 1.37.0 brought to you by InfluxData the makers of InfluxDB
2025-12-22T19:36:25Z I! Available plugins: 243 inputs, 9 aggregators, 35 processors, 26 parsers, 67 outputs, 8 secret-stores
2025-12-22T19:36:25Z I! Loaded inputs: cpu file mem sensors
2025-12-22T19:36:25Z I! Loaded aggregators: merge
2025-12-22T19:36:25Z I! Loaded processors: filter
2025-12-22T19:36:25Z I! Loaded secretstores:
2025-12-22T19:36:25Z W! Outputs are not used in testing mode!
2025-12-22T19:36:25Z I! Tags enabled: host=qdb
2025-12-22T19:36:25Z W! [agent] The default value of 'skip_processors_after_aggregators' will change to 'true' with Telegraf v1.40.0! If you need the current default behavior, please explicitly set the option to 'false'!
2025-12-22T19:36:25Z D! [agent] Initializing plugins
2025-12-22T19:36:25Z D! [agent] Starting service inputs
2025-12-22T19:36:25Z D! [parsers.xpath_json::file] Number of configs: 1
2025-12-22T19:36:25Z D! [parsers.xpath_json::file] Number of selected metric nodes: 1
2025-12-22T19:36:25Z D! [aggregators.merge] Updated aggregation range [2025-12-22 19:36:00 +0000 UTC, 2025-12-22 19:36:30 +0000 UTC]
2025-12-22T19:36:25Z D! [parsers.xpath_json::file] Number of configs: 1
2025-12-22T19:36:25Z D! [parsers.xpath_json::file] Number of selected metric nodes: 1
2025-12-22T19:36:26Z D! [agent] Stopping service inputs
2025-12-22T19:36:26Z D! [agent] Input channel closed
2025-12-22T19:36:26Z D! [agent] Processor channel closed
2025-12-22T19:36:26Z D! [aggregators.merge] Updated aggregation range [2025-12-22 19:36:00 +0000 UTC, 2025-12-22 19:36:30 +0000 UTC]
2025-12-22T19:36:26Z D! [agent] Aggregator channel closed
2025-12-22T19:36:26Z D! [agent] Processor channel closed
2025-12-22T19:36:26Z D! [agent] Stopped Successfully
> system-nvme,device=/dev/nvme1,host=qdb,model_name=XXYYY,serial_number=AABBZZ available_spare=100i,capacity=-9223372036854775808i,controller_busy_time=651i,critical_comp_time=0i,data_units_read=1825978i,data_units_written=8109889i,id=-9223372036854775808i,logical_block_size=512i,media_errors=0i,num_err_log_entries=0i,percentage_used=1i,power_cycles=61i,power_on_hours=3871i,temperature=32i,unsafe_shutdowns=43i,utilization=-9223372036854775808i,warning_temp_time=0i 1766432186000000000
```

### System info

Telegraf 1.37.0, running in docker

### Docker

_No response_

### Steps to reproduce

1. `smartctl -j -a /dev/nvme0 > nvme0.json`
2. Put the json next to `telegraf.conf` and start it
3. It fails to parse the JSON array `nvme_namespaces`. There's only one element in the array and I cannot address it to get `capacity/bytes` and `utilization/bytes`.

### Expected behavior

Using `--test` it finds these:

```
capacity=960197124096i
id=1i
utilization=86638583808i
```

### Actual behavior

These are actual values from JSON.
```
capacity=-9223372036854775808i
id=-9223372036854775808i
utilization=-9223372036854775808i
```

### Additional info

Here's the smartctl JSON:
[nvme0.json](https://github.com/user-attachments/files/24298904/nvme0.json)

I have tried with the following variant too, but without luck.
```
capacity = "number(/nvme_namespaces[1]/capacity/bytes)"
utilization = "number(/nvme_namespaces[1]/utilization/bytes)"

```

After some more research, I tried with this and it worked!
```
capacity = "number(//*[local-name()='nvme_namespaces']/*[1]/capacity/bytes)"
utilization = "number(//*[local-name()='nvme_namespaces']/*[1]/utilization/bytes)"
```

Is it a bug that the simpler one isn't working?

Contributor guide

Open the contributing guide

Research direction

Start with the xpath_json parser and the inputs.file XPath configuration, reproducing the issue with the supplied smartctl JSON and expressions for nvme_namespaces. Determine why indexed array paths return the minimum integer value while the local-name() workaround succeeds; done means the simpler expressions return the expected capacity, id, and utilization values.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
observability
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.