Azure / Azure/fluentd-plugin-mdsd

Timestamp sub-second resolution is lost in the to_msgpack

Open
#63 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
28
Forks
26
PR merge metrics
No merged PRs in 30d

Description

Sub-second resolution is supported after fluentd v0.14. However, the to_msgpack will only keep the unix second.

https://github.com/Azure/fluentd-plugin-mdsd/blob/9bacb00a722a6a27e44751652918c6262515184a/src/fluent-plugin-mdsd/lib/fluent/plugin/out_mdsd.rb#L72

A workaround is to us to_msgpack_ext to serialize the time.
```
[tag, time.to_msgpack_ext, record].to_msgpack
```
And de-serialize it in the emit.
```ruby
def write(chunk)
if use_source_timestamp
chunk.msgpack_each {|(tag, time, record)|
# Ruby (version >= 1.9) hash preserves insertion order. So the following item is
# the last item when iterating the 'record' hash.
record[emit_timestamp_name] = Time.at(Fluent::EventTime.from_msgpack_ext(time).to_r)
handle_record(tag, record)
}
else
chunk.msgpack_each {|(tag, record)|
record[emit_timestamp_name] = Time.now
handle_record(tag, record)
}
end
@log.flush
end
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.