Java APM agent produces invalid json with infinity values in histogram metrics
- Dominant language
- Gherkin
- Stars
- 427
- Forks
- 125
- PR merge metrics
- No merged PRs in 30d
Description
**Describe the bug**
In my application I wanted to upgrade to the latest java apm agent version available (to date 1.55.1). But this leads to problems with invalid json around histogram metrics and infinity values. Problem that occurs with histograms:
2025-11-27 09:24:10,260 [elastic-apm-server-reporter] WARN co.elastic.apm.agent.report.AbstractIntakeApiHandler - Response body: {
"accepted": 329,
"errors": [
{
"message": "decode error: data read error: v2.metricsetRoot.Metricset: v2.metricset.Samples: v2.metricsetSampleValue.Values: []float64: readNumberAsString: invalid number, error found in #10 byte of ...|5105400.0,\"Infinity\"|..., bigger context ...|000.0,3843071682022823400.0,4227378850225105400.0,\"Infinity\"],\"counts\":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,|...",
"document": "{\"metricset\":{\"timestamp\":1764235442476000,\"samples\":{\"process_uptime\":{\"value\":390.378},\"avg_statistics_data\":{\"value\":0.0},\"processing_times_last_5_mins.histogram\":{\"values\":[1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,16.0,21.0,26.0,31.0,36.0,41.0,46.0,51.0,56.0,64.0,85.0,106.0,127.0,148.0,169.0,190.0,211.0,232.0,256.0,341.0,426.0,511.0,596.0,681.0,766.0,851.0,936.0,1024.0,1365.0,1706.0,2047.0,2388.0,2729.0,3070.0,3411.0,3752.0,4096.0,5461.0,6826.0,8191.0,9556.0,10921.0,12286.0,13651.0,15016.0,16384.0,21845.0,27306.0,32767.0,38228.0,43689.0,49150.0,54611.0,60072.0,65536.0,87381.0,109226.0,131071.0,152916.0,174761.0,196606.0,218451.0,240296.0,262144.0,349525.0,436906.0,524287.0,611668.0,699049.0,786430.0,873811.0,961192.0,1048576.0,1398101.0,1747626.0,2097151.0,2446676.0,2796201.0,3145726.0,3495251.0,3844776.0,4194304.0,5592405.0,6990506.0,8388607.0,9786708.0,11184809.0,12582910.0,13981011.0,15379112.0,16777216.0,22369621.0,27962026.0,33554431.0,39146836.0,44739241.0,50331646.0,55924051.0,61516456.0,67108864.0,89478485.0,111848106.0,134217727.0,156587348.0,178956969.0,201326590.0,223696211.0,246065832.0,268435456.0,357913941.0,447392426.0,536870911.0,626349396.0,715827881.0,805306366.0,894784851.0,984263336.0,1073741824.0,1431655765.0,1789569706.0,2147483647.0,2505397588.0,2863311529.0,3221225470.0,3579139411.0,3937053352.0,4294967296.0,5726623061.0,7158278826.0,8589934591.0,10021590356.0,11453246121.0,12884901886.0,14316557651.0,15748213416.0,17179869184.0,22906492245.0,28633115306.0,34359738367.0,40086361428.0,45812984489.0,51539607550.0,57266230611.0,62992853672.0,68719476736.0,91625968981.0,114532461226.0,137438953471.0,160345445716.0,183251937961.0,206158430206.0,229064922451.0,251971414696.0,274877906944.0,366503875925.0,458129844906.0,549755813887.0,641381782868.0,733007751849.0,824633720830.0,916259689811.0,1007885658792.0,1099511627776.0,1466015503701.0,1832519379626.0,2199023255551.0,2565527131476.0,2932031007401.0,3298534883326.0,3665038759251.0,4031542635176.0,4398046511104.0,5864062014805.0,7330077518506.0,8796093022207.0,10262108525908.0,11728124029609.0,13194139533310.0,14660155037011.0,16126170540712.0,17592186044416.0,23456248059221.0,29320310074026.0,35184372088831.0,41048434103636.0,46912496118441.0,52776558133246.0,58640620148051.0,64504682162856.0,70368744177664.0,93824992236885.0,117281240296106.0,140737488355327.0,164193736414548.0,187649984473769.0,211106232532990.0,234562480592211.0,258018728651432.0,281474976710656.0,375299968947541.0,469124961184426.0,562949953421311.0,656774945658196.0,750599937895081.0,844424930131966.0,938249922368851.0,1032074914605736.0,1125899906842624.0,1501199875790165.0,1876499844737706.0,2251799813685247.0,2627099782632788.0,3002399751580329.0,3377699720527870.0,3752999689475411.0,4128299658422952.0,4503599627370496.0,6004799503160661.0,7505999378950826.0,9007199254740991.0,10508399130531156.0,12009599006321320.0,13510798882111486.0,15011998757901652.0,16513198633691816.0,18014398509481984.0,24019198012642644.0,30023997515803304.0,36028797018963970.0,4.2033596522124624E16,48038396025285290.0,54043195528445950.0,60047995031606610.0,66052794534767270.0,72057594037927940.0,96076792050570580.0,120095990063213230.0,144115188075855870.0,168134386088498530.0,192153584101141150.0,216172782113783800.0,240191980126426460.0,264211178139069100.0,288230376151711740.0,384307168202282300.0,480383960252852900.0,576460752303423500.0,672537544353994100.0,768614336404564600.0,864691128455135200.0,960767920505705900.0,1056844712556276400.0,1152921504606847000.0,1537228672809129200.0,1921535841011411700.0,2305843009213694000.0,2690150177415976400.0,3074457345618258400.0,3458764513820541000.0,3843071682022823400.0,4227378850225105400.0,\"Infinity\"],\"counts\":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],\"type\":\"histogram\"},\"processing_times_statistics_last_5_mins.histogram\":{\"values\":[1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,16.0,21.0,26.0,31.0,36.0,41.0,46.0,51.0,56.0,64.0,85.0,106.0,127.0,148.0,169.0,190.0,211.0,232.0,256.0,341.0,426.0,511.0,596.0,681.0,766.0,851.0,936.0,1024.0,1365.0,1706.0,2047.0,2388.0,2729.0,3070.0,3411.0,3752.0,4096.0,5461.0,6826.0,8191.0,9556.0,10921.0,12286.0,13651.0,15016.0,16384.0,21845.0,27306.0,32767.0,38228.0,43689.0,49150.0,54611.0,60072.0,65536.0,87381.0,109226.0,131071.0,152916.0,174761.0,196606.0,218451.0,240296.0,262144.0,349525.0,436906.0,524287.0,611668.0,699049.0,786430.0,873811.0,961192.0,1048576.0,1398101.0,1747626.0,2097151.0,2446676.0,2796201.0,3145726.0,3495251.0,3844776.0,4194304.0,5592405.0,6990506.0,8388607.0,9786708.0,11184809.0,12582910.0,13981011.0,15379112.0,16777216.0,22369621.0,27962026.0,33554431.0,39146836.0,44739241.0,50331646.0,55924051.0,61516456.0,67108864.0,89478485.0,111848106.0,134217727.0,156587348.0,178956969.0,201326590.0,223696211.0,246065832.0,268435456.0,357913941.0,447392426.0,536870911.0,626349396.0,715827881.0,805306366.0,894784851.0,984263336.0,1073741824.0,1431655765.0,1789569706.0,2147483647.0,2505397588.0,2863311529.0,3221225470.0,3579139411.0,3937053352.0,4294967296.0,5726623061.0,7158278826.0,8589934591.0,10021590356.0,11453246121.0,12884901886.0,14316557651.0,15748213416.0,17179869184.0,22906492245.0,28633115306.0,34359738367.0,40086361428.0,45812984489.0,51539607550.0,57266230611.0,62992853672.0,68719476736.0,91625968981.0,114532461226.0,137438953471.0,160345445716.0,183251937961.0,206158430206.0,229064922451.0,251971414696.0,274877906944.0,366503875925.0,458129844906.0,549755813887.0,641381782868.0,733007751849.0,824633720830.0,916259689811.0,1007885658792.0,1099511627776.0,1466015503701.0,1832519379626.0,2199023255551.0,2565527131476.0,2932031007401.0,3298534883326.0,3665038759251.0,4031542635176.0,4398046511104.0,5864062014805.0,7330077518506.0,8796093022207.0,10262108525908.0,11728124029609.0,13194139533310.0,14660155037011.0,16126170540712.0,17592186044416.0,23456248059221.0,29320310074026.0,35184372088831.0,41048434103636.0,46912496118441.0,52776558133246.0,58640620148051.0,64504682162856.0,70368744177664.0,93824992236885.0,117281240296106.0,140737488355327.0,164193736414548.0,187649984473769.0,211106232532990.0,234562480592211.0,258018728651432.0,281474976710656.0,375299968947541.0,469124961184426.0,562949953421311.0,656774945658196.0,750599937895081.0,844424930131966.0,938249922368851.0,1032074914605736.0,1125899906842624.0,1501199875790165.0,1876499844737706.0,2251799813685247.0,2627099782632788.0,3002399751580329.0,3377699720527870.0,3752999689475411.0,4128299658422952.0,4503599627370496.0,6004799503160661.0,7505999378950826.0,9007199254740991.0,10508399130531156.0,12009599006321320.0,13510798882111486.0,15011998757901652.0,16513198633691816.0,18014398509481984.0,24019198012642644.0,30023997515803304.0,36028797018963970.0,4.2033596522124624E16,48038396025285290.0,54043195528445950.0,60047995031606610.0,66052794534767270.0,72057594037927940.0,96076792050570580.0,120095990063213230.0,144115188075855870.0,168134386088498530.0,192153584101141150.0,216172782113783800.0,240191980126426460.0,264211178139069100.0,288230376151711740.0,384307168202282300.0,480383960252852900.0,576460752303423500.0,672537544353994100.0,768614336404564600.0,864691128455135200.0,960767920505705900.0,1056844712556276400.0,1152921504606847000.0,1537228672809129200.0,1921535841011411700.0,2305843009213694000.0,2690150177415976400.0,3074457345618258400.0,3458764513820541000.0,3843071682022823400.0,4227378850225105400.0,\"Infinity\"],\"counts\":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],\"type\":\"histogram\"}}}}"
}
]
}
"Infinity" is coming from the “+Inf” bucket that Micrometer uses for timers/distribution summaries.
JSON does not allow Infinity / NaN as numeric values, and the APM Server’s decoder expects these to be valid float64. As soon as it sees "Infinity" it fails with:
v2.metricsetSampleValue.Values: []float64: readNumberAsString: invalid number
**To Reproduce**
1. Use Java APM Agent version 1.55.1
2. Have histogram percentile publishing enabled in your metric configuration. E.g. like this:
return DistributionSummary.builder(name)
.description(description)
.publishPercentileHistogram()
.publishPercentiles(0.0, 0.99) // min value, 99th percentile
.distributionStatisticExpiry(Duration.ofMinutes(minutesToMonitor))
.register(registry);
3. Then start your application and make sure your metric gets recorded.
4. Error with invalid json will occur.
**Expected behavior**
No invalid json should be produced by apm agent.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.