influxdata / influxdata/telegraf

outputs.postgresql: nil pointer panic in sqltemplate.Template.Render when a metric has zero tags

Open
#19,501 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
17.8k
Forks
5.8k
Avg merge
1d 20h
Merged PRs (30d)
161

Description

### Telegraf version

1.39.3 (also observed identically across `CREATE TABLE` and `ALTER TABLE ADD COLUMN` code paths, so likely present in earlier 1.39.x/1.38.x too — not yet tested on those).

### Platform

RHEL/AlmaLinux, PostgreSQL + TimescaleDB backend (`192.168.30.4:5432`, `timescale` database, `lab_data` schema).

### Summary

`outputs.postgresql` panics with a nil pointer dereference inside `sqltemplate.(*Template).Render` whenever the metric being written resolves to **zero tag columns** (i.e. only a timestamp column plus field columns, no tag-derived columns at all). This reproduces identically whether the target table already exists (hitting the `ADD COLUMN` path) or does not exist yet (hitting the `CREATE TABLE` path), and is independent of column count or field type mix. As soon as the metric carries at least one tag, the exact same write succeeds (or fails gracefully with an ordinary SQL error) instead of panicking.

### Panic

```
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x6b5f965]
goroutine 256 [running]:
github.com/influxdata/telegraf/plugins/outputs/postgresql/sqltemplate.(*Template).Render(0x35b7932dd580, 0x0, {0x35b792dc2780, 0x9, 0x35b793470f40?}, 0x0, 0x0)
/go/src/github.com/influxdata/telegraf/plugins/outputs/postgresql/sqltemplate/template.go:397 +0x385
github.com/influxdata/telegraf/plugins/outputs/postgresql.(*TableManager).update(0x35b793442ed0, {0xb51cf48, 0x35b7942a86e0}, {0xb5881b0, 0x35b7927b55a0}, 0x35b793442480?, {0x35b7932fc2a0, 0x2, 0x48b4b9?}, {0x35b792dc2780, ...}, ...)
/go/src/github.com/influxdata/telegraf/plugins/outputs/postgresql/table_manager.go:383 +0xac5
github.com/influxdata/telegraf/plugins/outputs/postgresql.(*TableManager).EnsureStructure(0x35b793442ed0, {0xb51cf48, 0x35b7942a86e0}, {0x7feccc6691c0, 0x35b7930d8030}, 0x35b793442480, {0x35b792dc2600, 0x9, 0x9}, {0x35b7932fc2a0, ...}, ...)
/go/src/github.com/influxdata/telegraf/plugins/outputs/postgresql/table_manager.go:292 +0x17b2
github.com/influxdata/telegraf/plugins/outputs/postgresql.(*TableManager).MatchSource(0x35b793442ed0, {0xb51cf48, 0x35b7942a86e0}, {0x7feccc6691c0, 0x35b7930d8030}, 0x35b793200000)
/go/src/github.com/influxdata/telegraf/plugins/outputs/postgresql/table_manager.go:115 +0xc85
github.com/influxdata/telegraf/plugins/outputs/postgresql.(*Postgresql).writeMetricsFromMeasure(0x35b7932f58c8, {0xb51cf48, 0x35b7942a86e0}, {0x7feccc6691c0, 0x35b7930d8030}, 0x35b793200000)
/go/src/github.com/influxdata/telegraf/plugins/outputs/postgresql/postgresql.go:432 +0x56
github.com/influxdata/telegraf/plugins/outputs/postgresql.(*Postgresql).writeSequential(0x35b7932f58c8, 0x35b793442090)
/go/src/github.com/influxdata/telegraf/plugins/outputs/postgresql/postgresql.go:291 +0x2ac
github.com/influxdata/telegraf/plugins/outputs/postgresql.(*Postgresql).Write(0x35b7932f58c8, {0x35b7932fc030?, 0xc78631?, 0x35b793465e88?})
/go/src/github.com/influxdata/telegraf/plugins/outputs/postgresql/postgresql.go:259 +0xbc
github.com/influxdata/telegraf/models.(*RunningOutput).writeMetrics(0x35b79265a6c0, {0x35b7932fc030, 0x1, 0x1})
/go/src/github.com/influxdata/telegraf/models/running_output.go:397 +0x114
github.com/influxdata/telegraf/models.(*RunningOutput).doTransaction(0x35b79265a6c0)
/go/src/github.com/influxdata/telegraf/models/running_output.go:377 +0x55
github.com/influxdata/telegraf/models.(*RunningOutput).Write(0x35b79265a6c0)
/go/src/github.com/influxdata/telegraf/models/running_output.go:341 +0x3c6
github.com/influxdata/telegraf/agent.(*Agent).flushOnce.func1()
/go/src/github.com/influxdata/telegraf/agent/agent.go:922 +0x23
created by github.com/influxdata/telegraf/agent.(*Agent).flushOnce in goroutine 253
/go/src/github.com/influxdata/telegraf/agent/agent.go:921 +0x9e
```

### Relevant config

`telegraf.conf` (agent + global pipeline + output, trimmed to the relevant parts):

```toml
[agent]
interval = "60s"
round_interval = true
flush_interval = "10s"
skip_processors_after_aggregators = false

# Derive final_measurement tag from hostname, then promote it to the
# measurement name — the intended design is "one wide row per host",
# with per-metric identity encoded into field names rather than tags.
[[processors.template]]
namepass = ["*"]
order = 140
tag = "final_measurement"
template = '{{ .Tag "host" }}'

[[processors.strings]]
namepass = ["*"]
order = 141
[[processors.strings.lowercase]]
tag = "final_measurement"

[[processors.converter]]
namepass = ["*"]
order = 160
[processors.converter.tags]
measurement = ["final_measurement"]

[[aggregators.merge]]
namepass = ["*"]
order = 200
period = "60s"
round_timestamp_to = "1s"
drop_original = true

[[outputs.postgresql]]
connection = "postgres://telegraf:***@192.168.30.4:5432/timescale?sslmode=disable"
schema = "lab_data"
tags_as_foreign_keys = false
timestamp_column_name = "time"
timestamp_column_type = "timestamp without time zone"
create_templates = [
'''CREATE TABLE {{ .table }} ({{ .columns }})''',
'''SELECT create_hypertable({{ .table|quoteLiteral }}, 'time', chunk_time_interval => INTERVAL '7d')'''
]
add_column_templates = [
'''ALTER TABLE {{ .table }} ADD COLUMN IF NOT EXISTS {{ .columns|join ", ADD COLUMN IF NOT EXISTS " }}''',
]
uint64_type = "numeric"
log_level = "warn"
```

`nvidia_smi.conf` (the input used to isolate the bug — its pipeline is deliberately designed to strip every tag, encoding GPU index into the field name instead):

```toml
[[inputs.nvidia_smi]]
name_override = "smi"
tagexclude = ["arch", "compute_mode", "name", "pstate", "type", "uuid"]
fieldexclude = [ /* full raw-field exclude list trimmed for brevity */ ]

[[processors.template]]
namepass = ["smi"]
order = 10
tag = '{{printf "gpu%s_watt" (.Tag "index")}}'
template = '{{ .Field "power_draw" }}'

[[processors.template]]
namepass = ["smi"]
order = 11
tag = '{{printf "gpu%s_celsius" (.Tag "index")}}'
template = '{{ .Field "temperature_gpu" }}'

[[processors.converter]]
namepass = ["smi"]
order = 12
[processors.converter.tags]
float = ["gpu*_watt", "gpu*_celsius"]

[[processors.override]]
namepass = ["smi"]
order = 150
tagexclude = ["host", "index"] # <-- this is what strips the metric to zero tags
fieldexclude = ["power_draw", "temperature_gpu"]
```

With the `tagexclude = ["host", "index"]` override at order 150 in place, the final merged metric for measurement `smi` (promoted to the hostname's measurement/table name at order 160) carries **only fields** — `gpu0_watt`, `gpu0_celsius`, `gpu1_watt`, ... — and no tags whatsoever.

### Steps to reproduce / isolation performed

| # | Table state | Metric shape reaching output | Result |
|---|---|---|---|
| 1 | Fresh table, full input fleet (`cpu_power`, `drive_temps`, `ethtool`, `ipmi_sensor`, `nvidia_smi`, `nxp`) | first flush's metric (2 columns) | **panic**, identical trace |
| 2 | Fresh table, `nvidia_smi.conf` alone, stale `fieldexclude` (schema drift left ~19 extra raw fields unfiltered) | 28 columns, 0 tags | **panic**, identical trace |
| 3 | Fresh table, `nvidia_smi.conf` alone, `fieldexclude` corrected so only intended fields reach output | 9 columns (`gpu0..3_watt/celsius` + `host`... actually 0 tags, 8 fields), 0 tags | **panic**, identical trace |
| 4 | Table pre-created manually (bare: just `time` + `create_hypertable`), same corrected config | same 9-column write via `ALTER TABLE ADD COLUMN` instead of `CREATE TABLE` | **panic**, identical trace |
| 5 | Table dropped and allowed to be created fresh by Telegraf, order-150 `tagexclude` removed so `host`/`index` survive as tags | same fields, but now 2 tag columns present | **no panic** — either succeeds, or fails with an ordinary SQL error (e.g. a table-ownership permission error), logged normally |

Across all 5 attempts the only variable that correlated with panic vs. no-panic was whether the metric had at least one tag column. Column count (2, 9, 28), field type mix, and whether the table pre-existed all varied independently without affecting the outcome.

### Expected behavior

A metric with zero tags (all identity encoded into field names, only a timestamp + field columns) is a valid, if unusual, shape and should be handled the same way a tag-bearing metric is — either written successfully or failed with a normal SQL/config error, not a panic that kills the whole Telegraf process.

### Workaround

Ensure at least one tag (e.g. `host`) survives on the final metric before it reaches `outputs.postgresql` — i.e. don't strip every tag in a per-input cleanup processor. This avoids the panic but is a workaround, not a fix, since it forces a redundant tag column purely to dodge the crash.

Contributor guide

Open the contributing guide

Research direction

Start at plugins/outputs/postgresql/sqltemplate/template.go:397 and follow the call from TableManager.update in plugins/outputs/postgresql/table_manager.go. Reproduce with a metric containing fields but zero tags, using both the CREATE TABLE and ALTER TABLE paths described in the issue. Done means the metric no longer causes a nil-pointer panic and instead succeeds or reports a normal SQL/configuration error.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, postgresql
Domain
backend, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.