redpanda-data / redpanda-data/benthos
[BUG] Precision issue using ts_format and ts_strftime in bloblang
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 571
- Forks
- 120
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 18
Description
I have encountered a precision issue using ts_format in bloblang mapping.
Input
root.time = (1555234838050/1000).ts_format("2006-01-02T15:04:05.999-07:00", "UTC")
Expected output
{
"time": "2019-04-14T09:40:38.050+00:00"
}
Actual output
{
"time": "2019-04-14T09:40:38.049+00:00"
}
Version
4.53.0
The same problem occurs using ts_strftime(). I read that these two methods support nanosecond precision, so ms precision shouldn't be a problem.
PS: I currently use a dirty trick to fix this problem and avoiding heavily string operation at the same time (which is critical for my high frequency use case):
root.time = (1555234838050/1000+0.0004).ts_format("2006-01-02T15:04:05.999-07:00", "UTC")
But I guess this can't be 100% safe as trying to build a feature on a bug.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the issue's Bloblang mapping with ts_format and ts_strftime using the provided millisecond timestamp and compare the actual and expected outputs. Trace the implementation or entry points for these two methods; done means both preserve the expected millisecond precision without relying on the timestamp offset workaround.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100