prometheus / prometheus/common
MetricFamilyToOpenMetrics produces output that cannot be parsed
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 295
- Forks
- 367
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 18
Description
I am debugging https://github.com/prometheus/jmx_exporter/issues/621, and I found I can reproduce the root cause of the error with the following test in github.com/prometheus/common/expfmt:
package expfmt
import (
"bytes"
"strings"
"testing"
)
func TestJvmClassesLoaded(t *testing.T) {
in := `
# HELP jvm_classes_loaded The number of classes that are currently loaded in the JVM
# TYPE jvm_classes_loaded gauge
jvm_classes_loaded 6129.0
# HELP jvm_classes_loaded_total The total number of classes that have been loaded since the JVM has started execution
# TYPE jvm_classes_loaded_total counter
jvm_classes_loaded_total 6143.0
`
metricFamilies, err := parser.TextToMetricFamilies(strings.NewReader(in))
if err != nil {
t.Error(err)
}
openMetricsOutput := bytes.NewBuffer(make([]byte, 0))
for _, metricFamily := range metricFamilies {
if _, err := MetricFamilyToOpenMetrics(openMetricsOutput, metricFamily); err != nil {
t.Fatal(err)
}
}
if _, err := FinalizeOpenMetrics(openMetricsOutput); err != nil {
t.Fatal(err)
}
_, err = parser.TextToMetricFamilies(strings.NewReader(openMetricsOutput.String()))
if err != nil {
t.Error(err)
}
}
It fails with
=== RUN TestJvmClassesLoaded
jvm_classes_loaded_test.go:33: text format parsing error in line 4: second HELP line for metric name "jvm_classes_loaded"
--- FAIL: TestJvmClassesLoaded (0.00s)
FAIL
FAIL github.com/prometheus/common/expfmt 0.002s
FAIL
As the error message indicates, the reason is that the _total suffix is truncated in HELP and TYPE (as defined in #214), and the resulting "short name" conflicts with the name of the gauge.
Two full serialization steps are needed to trigger the error (OpenMetrics format -> Go objects -> OpenMetrics format -> Go objects), so this might seem like an issue that doesn't occur in practice. However, in https://github.com/prometheus/jmx_exporter/issues/621 this happens because of the following setup:
jmx_exporter -> exporter_exporter -> Prometheus server
I'm not sure what to do about this. Should exporters prevent metrics from having the same "short name"?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in the expfmt package with MetricFamilyToOpenMetrics, FinalizeOpenMetrics, and parser.TextToMetricFamilies, using the reproducer in the issue as the first test to run. Trace how the _total suffix is handled during serialization and parsing. Done means the reported OpenMetrics round trip no longer produces the conflicting HELP error, with the handling of duplicate short names clarified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100