influxdata / influxdata/telegraf

[[processors.converter]] Support multiple timestamps

Open
#17,756 0 comments 0 reactions 0 assignees View on GitHub
feature request
Dominant language
Go
Stars
17.8k
Forks
5.8k
Avg merge
1d 20h
Merged PRs (30d)
161

Description

### Use Case

I'm sending metrics via JSON that contain multiple timestamp fields in RFC3339Nano format. While the primary metric timestamp (metric_time) works correctly when configured as timestamp_key in the JSON parser, additional timestamp fields (e.g., create_time, update_time) need to be stored in PostgreSQL timestamp columns.

Example JSON
```json
{
"entity": 1,
"create_time": "2025-10-02T21:31:03.965716Z",
"metric_time": "2025-10-02T21:31:03.965716Z",
"metric": "test_metric"
}
```
This can be taken as an improvement to
https://github.com/influxdata/telegraf/issues/12746

### Expected behavior

Certain configured fields(apart from metric time) should be stored as timestamp in postgres.

### Actual behavior

When trying to store create_time (an RFC3339Nano string) into a PostgreSQL timestamp with time zone column, the PostgreSQL output plugin fails with:
```
ERROR: incorrect binary data format (SQLSTATE 22P03)
```
This occurs because:
- The JSON parser treats create_time as a regular string field
- PostgreSQL's binary COPY FROM expects time.Time objects for timestamp columns
- The string field fails type validation during binary copy

### Additional info

We can enhance following options
- Update processors.converter to support additional timestamps
```
[[processors.converter]]
[processors.converter.fields]
timestamp_field = ["create_time"] # NEW: Different from 'timestamp'
timestamp_format = "2006-01-02T15:04:05.999999999Z07:00"
```

- Add a configuration option to postgres plugin to specify which fields should be parsed as timestamps
```
[[outputs.postgresql]]
connection = "..."
timestamp_column_name = "metric_time"
timestamp_column_type = "timestamp with time zone"

# NEW: Additional timestamp field mappings
[outputs.postgresql.timestamp_fields]
update_time = "2006-01-02T15:04:05.999999999Z07:00"
create_time = "2006-01-02T15:04:05.999999999Z07:00"
```

Contributor guide

Open the contributing guide

Research direction

Start by reading the processors.converter entry point and the PostgreSQL output plugin to understand how configured fields and timestamp columns are currently handled. Compare the existing metric timestamp path with the proposed additional-field mappings, then define and test one consistent configuration approach. Done means configured RFC3339Nano fields reach PostgreSQL as timestamp values without binary COPY errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, postgresql
Domain
backend, database
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.