influxdata / influxdata/telegraf
[[processors.converter]] Support multiple timestamps
- Dominant language
- Go
- Stars
- 17.8k
- Forks
- 5.8k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 161
Description
### Use Case
I'm sending metrics via JSON that contain multiple timestamp fields in RFC3339Nano format. While the primary metric timestamp (metric_time) works correctly when configured as timestamp_key in the JSON parser, additional timestamp fields (e.g., create_time, update_time) need to be stored in PostgreSQL timestamp columns.
Example JSON
```json
{
"entity": 1,
"create_time": "2025-10-02T21:31:03.965716Z",
"metric_time": "2025-10-02T21:31:03.965716Z",
"metric": "test_metric"
}
```
This can be taken as an improvement to
https://github.com/influxdata/telegraf/issues/12746
### Expected behavior
Certain configured fields(apart from metric time) should be stored as timestamp in postgres.
### Actual behavior
When trying to store create_time (an RFC3339Nano string) into a PostgreSQL timestamp with time zone column, the PostgreSQL output plugin fails with:
```
ERROR: incorrect binary data format (SQLSTATE 22P03)
```
This occurs because:
- The JSON parser treats create_time as a regular string field
- PostgreSQL's binary COPY FROM expects time.Time objects for timestamp columns
- The string field fails type validation during binary copy
### Additional info
We can enhance following options
- Update processors.converter to support additional timestamps
```
[[processors.converter]]
[processors.converter.fields]
timestamp_field = ["create_time"] # NEW: Different from 'timestamp'
timestamp_format = "2006-01-02T15:04:05.999999999Z07:00"
```
- Add a configuration option to postgres plugin to specify which fields should be parsed as timestamps
```
[[outputs.postgresql]]
connection = "..."
timestamp_column_name = "metric_time"
timestamp_column_type = "timestamp with time zone"
# NEW: Additional timestamp field mappings
[outputs.postgresql.timestamp_fields]
update_time = "2006-01-02T15:04:05.999999999Z07:00"
create_time = "2006-01-02T15:04:05.999999999Z07:00"
```
Contributor guide
Research direction
Start by reading the processors.converter entry point and the PostgreSQL output plugin to understand how configured fields and timestamp columns are currently handled. Compare the existing metric timestamp path with the proposed additional-field mappings, then define and test one consistent configuration approach. Done means configured RFC3339Nano fields reach PostgreSQL as timestamp values without binary COPY errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, postgresql
- Domain
- backend, database
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100