POJO-ification of Metrics
- Dominant language
- Java
- Stars
- 14.9k
- Forks
- 3.5k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 88
Description
(Jake)
Hijacking original comment with break down of progress:
- [x] General acceptance of `Witness` direction (see below)
- [x] Initial `Witness` data model https://github.com/elastic/logstash/pull/7947
- [x] Metrics clean up in prep for `Witness` https://github.com/elastic/logstash/pull/7872
- [x] Jackson Serialization for `Witness` https://github.com/elastic/logstash/pull/7984
- [x] Remove lazy serialization of Metrics https://github.com/elastic/logstash/issues/7885
- [x] Clean up and support for DLQ and PQ via `Witness` https://github.com/elastic/logstash/pull/8023
- [x] Support for Plugin Metrics via `Witness` https://github.com/elastic/logstash/pull/8053
- [x] General approval to remove the namespace API https://github.com/elastic/logstash/issues/7971
- [ ] Ruby - Complete removal of namespace API in favor of `Witness` (for :stats) https://github.com/elastic/logstash/pull/8062
- [ ] Model JVM / OS stats in Java with support for JSON serialization.
- [ ] Ruby - Complete removal of namespace API in favor of `Witness` (for :jvm, os, etc.)
- [ ] Ruby - use Java based serialization as data provider to Sinatra for JSON API
- [ ] XPack - update to use `Witness` API
- [ ] Improve the semantics of the API : https://github.com/elastic/logstash/issues/7973
Outside of scope for this issue:
* Move HTTP server from Ruby to Java
* Move data OS/Java stats data retrieval to Java. (discussion: https://github.com/elastic/logstash/issues/7978 and https://github.com/elastic/logstash/issues/8048)
----------------
(Andrew)
This issue sets out to clearly define our goals for the refactor of Logstash metrics. It takes over from #7274 .
Our current metrics implementation provides more than we need and has some performance problems, like #7772 to boot. Additionally, the design decision to use `Map` objects to store the hierarchy of metrics has resulted in some performance issues as well as prevented us from leaning on the JVM type checker to verify that metric accessess / stores are correct.
After numerous discussions with @jakelandis @ph and @jsvd we've decided to move toward a simpler POJO based architecture, where we will use the new `Gauge` and `Counter` types directly.
So, instead of ruby code like:
```ruby
class Foo
def initialize(external_metric_thing)
@metrics = external_metric_thing.namespace(:foo)
end
def receivedEvent
@metric.increment(:events_received, 1)
end
end
```
We will write something more like:
```ruby
class Foo
def initialize
self.metrics = FooMetrics.new()
end
def receivedEvent
self.fooMetrics.receivedEvent()
end
end
```
```java
class FooMetrics {
public FooMetrics() {
this.receivedEvents = new Counter()
}
public void receivedEvent() {
this.receivedEvents.increment()
}
}
```
These java metrics classes will be much more complicated in the future, and might increment multiple metrics based on a single lifecycle event, and even increment metrics in linked objects.
Contributor guide
Assessment
This issue has not been assessed yet.