elastic / elastic/logstash

Witness API improvements

Open
#7,973 2 comments 0 reactions 1 assignee Claimed by @jakelandis View on GitHub
discuss enhancement
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
19h 14m
Merged PRs (30d)
63

Description

With the introduction of a type safe fluent API (the `Witness` API) for the metrics #7788, we have the opportunity to make the the metrics read like prose.

For the initial draft, I simply mirrored the names used in the metrics namespace's.

For example:

`[:stats, :pipelines, :main, :plugins, :inputs, :"18ff6d6eee97a21c9a86ad55223c9d9211de1683-1", :events] out = 1`

The part in `[..]` is the namespace, and the part outside is the action/event/metric and value.

Mapped to the `Witness API`, this would be:

` Witness.instance().pipeline("main").inputs("18ff6d6eee97a21c9a86ad55223c9d9211de1683-1").events().out();`

As @andrewvc pointed out, we should strive for:

`Witness->noun->noun->noun->verb`

Where the chained parts are nouns and the chain terminator is expressed as verb.

Many of the metrics already read this way ... but others do not.

For example:

`[:stats, :pipelines, :main, :queue] type = memory`

Would read:

`Witness.instance().pipeline("main").queue().type("memory");`

Which does not follow the terminate in a verb convention.

This is a working issue to identify any improvements to the fluent API.

Below is a list of all existing namespaces and keys for a simple tcp input, not filters and a stdout. Not a comprehensive list, but a good starting point.

Stats
----------------
[:stats, :events] duration_in_millis = 411
[:stats, :events] filtered = 1
[:stats, :events] in = 0
[:stats, :events] in = 0[:stats, :pipelines, :main, :plugins, :inputs, :"18ff6d6eee97a21c9a86ad55223c9d9211de1683-2", :events] out = 0
[:stats, :events] in = 1
[:stats, :events] out = 1
[:stats, :events] queue_push_duration_in_millis = 17
[:stats, :pipelines, :main, :config] batch_delay = 5
[:stats, :pipelines, :main, :config] batch_size = 125
[:stats, :pipelines, :main, :config] config_reload_automatic = false
[:stats, :pipelines, :main, :config] config_reload_interval = 3000000000
[:stats, :pipelines, :main, :config] dead_letter_queue_enabled = false
[:stats, :pipelines, :main, :config] workers = 8
[:stats, :pipelines, :main, :events] duration_in_millis = 411
[:stats, :pipelines, :main, :events] filtered = 1
[:stats, :pipelines, :main, :events] in = 1
[:stats, :pipelines, :main, :events] out = 1
[:stats, :pipelines, :main, :events] queue_push_duration_in_millis = 17
[:stats, :pipelines, :main, :plugins, :codecs, :"18ff6d6eee97a21c9a86ad55223c9d9211de1683-3"] name = rubydebug
[:stats, :pipelines, :main, :plugins, :inputs, :"18ff6d6eee97a21c9a86ad55223c9d9211de1683-1", :events] out = 0
[:stats, :pipelines, :main, :plugins, :inputs, :"18ff6d6eee97a21c9a86ad55223c9d9211de1683-1", :events] out = 1
[:stats, :pipelines, :main, :plugins, :inputs, :"18ff6d6eee97a21c9a86ad55223c9d9211de1683-1", :events] queue_push_duration_in_millis = 0
[:stats, :pipelines, :main, :plugins, :inputs, :"18ff6d6eee97a21c9a86ad55223c9d9211de1683-1", :events] queue_push_duration_in_millis = 17
[:stats, :pipelines, :main, :plugins, :inputs, :"18ff6d6eee97a21c9a86ad55223c9d9211de1683-1"] name = tcp
[:stats, :pipelines, :main, :plugins, :inputs, :"18ff6d6eee97a21c9a86ad55223c9d9211de1683-2", :events] queue_push_duration_in_millis = 0
[:stats, :pipelines, :main, :plugins, :inputs, :"18ff6d6eee97a21c9a86ad55223c9d9211de1683-2"] name = tcp
[:stats, :pipelines, :main, :plugins, :outputs, :"18ff6d6eee97a21c9a86ad55223c9d9211de1683-4", :events] duration_in_millis = 357
[:stats, :pipelines, :main, :plugins, :outputs, :"18ff6d6eee97a21c9a86ad55223c9d9211de1683-4", :events] in = 1
[:stats, :pipelines, :main, :plugins, :outputs, :"18ff6d6eee97a21c9a86ad55223c9d9211de1683-4", :events] out = 1
[:stats, :pipelines, :main, :plugins, :outputs, :"18ff6d6eee97a21c9a86ad55223c9d9211de1683-4"] name = stdout
[:stats, :pipelines, :main, :queue] type = memory
[:stats, :pipelines, :main, :reloads] failures = 0
[:stats, :pipelines, :main, :reloads] last_error =
[:stats, :pipelines, :main, :reloads] last_failure_timestamp =
[:stats, :pipelines, :main, :reloads] last_success_timestamp =
[:stats, :pipelines, :main, :reloads] successes = 0
[:stats, :reloads] failures = 0
[:stats, :reloads] successes = 0

JVM
--------------------------
[:jvm, :gc, :collectors, :old] collection_count = 3
[:jvm, :gc, :collectors, :old] collection_time_in_millis = 482
[:jvm, :gc, :collectors, :young] collection_count = 11
[:jvm, :gc, :collectors, :young] collection_time_in_millis = 194
[:jvm, :memory, :heap] committed_in_bytes = 631767040
[:jvm, :memory, :heap] max_in_bytes = 4248305664
[:jvm, :memory, :heap] peak_max_in_bytes = 4325376000
[:jvm, :memory, :heap] peak_used_in_bytes = 315057600
[:jvm, :memory, :heap] used_in_bytes = 90805792
[:jvm, :memory, :heap] used_percent = 6
[:jvm, :memory, :non_heap] committed_in_bytes = 116457472
[:jvm, :memory, :non_heap] max_in_bytes = 1325400063
[:jvm, :memory, :non_heap] peak_max_in_bytes = 1325400063
[:jvm, :memory, :non_heap] peak_used_in_bytes = 107736168
[:jvm, :memory, :non_heap] used_in_bytes = 107014632
[:jvm, :memory, :pools, :old] committed_in_bytes = 362283008
[:jvm, :memory, :pools, :old] max_in_bytes = 2863661056
[:jvm, :memory, :pools, :old] peak_max_in_bytes = 2863661056
[:jvm, :memory, :pools, :old] peak_used_in_bytes = 68818016
[:jvm, :memory, :pools, :old] used_in_bytes = 68818016
[:jvm, :memory, :pools, :survivor] committed_in_bytes = 41943040
[:jvm, :memory, :pools, :survivor] max_in_bytes = 41943040
[:jvm, :memory, :pools, :survivor] peak_max_in_bytes = 41943040
[:jvm, :memory, :pools, :survivor] peak_used_in_bytes = 36862304
[:jvm, :memory, :pools, :survivor] used_in_bytes = 0
[:jvm, :memory, :pools, :young] committed_in_bytes = 227540992
[:jvm, :memory, :pools, :young] max_in_bytes = 1342701568
[:jvm, :memory, :pools, :young] peak_max_in_bytes = 1419771904
[:jvm, :memory, :pools, :young] peak_used_in_bytes = 209377280
[:jvm, :memory, :pools, :young] used_in_bytes = 21987776
[:jvm, :process, :cpu] load_average = {:"1m"=>3.4970703125}
[:jvm, :process, :cpu] percent = 18
[:jvm, :process, :cpu] total_in_millis = 59468
[:jvm, :process, :mem] total_virtual_in_bytes = 8738852864
[:jvm, :process] max_file_descriptors = 10240
[:jvm, :process] open_file_descriptors = 82
[:jvm, :process] peak_open_file_descriptors = 82
[:jvm, :threads] count = 6
[:jvm, :threads] peak_count = 6
[:jvm] uptime_in_millis = 41257

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.