airbytehq / airbytehq/airbyte

metrics-reporter emits nothing since 1.8.0 — Micronaut bean definitions are not generated

Aberta
#85,345 0 comentários 0 reações 0 responsáveis Ver no GitHub
autoteam community team/use
Linguagem predominante
Python
Estrelas
22.1k
Forks
5.3k
Métricas de merge de PRs
Métricas de PR pendentes

Descrição

# metrics-reporter emits nothing since 1.8.0 — Micronaut bean definitions are not generated (Kotlin module still uses `annotationProcessor` instead of `ksp`)

## Summary

`airbyte-metrics-reporter` has produced **no metrics at all** in every release from
**1.8.0 onward**, including 2.0.x, 2.1.x and 2.2.0. It starts cleanly, connects to
the database, starts its Micrometer OTLP publisher, and then does nothing — no
error, no warning, no metric.

Root cause: the module was converted from Java to Kotlin at 1.8.0, but its
`build.gradle.kts` still declares the **javac** annotation processor
(`annotationProcessor(...)`). Micronaut's processor therefore sees no sources, so
no `$Definition` classes and no `META-INF/micronaut/` index are produced. Micronaut
is compile-time DI, so at runtime none of the emitters exist as beans.

## Evidence

### The published jars stop containing bean definitions at 1.8.0

`io.airbyte.airbyte-metrics-reporter-.jar` from `airbyte/metrics-reporter:`:

| tag | `.class` files | `$Definition` classes | `META-INF/micronaut/` |
|---|---:|---:|---:|
| 0.63.8 | 36 | 36 | 18 |
| 1.0.0 | 36 | 36 | 18 |
| 1.5.1 | 35 | 32 | 16 |
| 1.6.0 | 35 | 32 | 16 |
| **1.7.0** | 35 | **32** | **16** |
| **1.8.0** | 18 | **0** | **0** |
| 2.0.0 | 18 | 0 | 0 |
| 2.0.1 | 18 | 0 | 0 |
| 2.1.0 | 18 | 0 | 0 |
| 2.1.1 | 18 | 0 | 0 |
| 2.2.0 | 18 | 0 | 0 |

From 1.8.0 the jar contains only the 18 raw Kotlin classes plus `application.yml`
and `micronaut-banner.txt`. No generated definitions, no `META-INF/micronaut/`.

### That boundary is exactly the Java → Kotlin conversion

| tag | source dir | `ksp(...)` in build.gradle.kts |
|---|---|---:|
| v1.7.0 | `src/main/java` | 0 |
| v2.0.0 | `src/main/kotlin` | 0 |
| main | `src/main/kotlin` | 0 |

Through 2.0.x, `airbyte-metrics/reporter/build.gradle.kts` declared:

```kotlin
annotationProcessor(platform(libs.micronaut.platform))
annotationProcessor(libs.bundles.micronaut.annotation.processor)
```

`annotationProcessor` is the javac configuration and does not apply to Kotlin
sources, so it was inert from the moment the module became Kotlin. On current
`main` those lines have been **removed entirely** and no `ksp(...)` was added,
so the module now has no annotation processor at all.

`annotationProcessor` is the javac configuration and does not apply to Kotlin
sources. Every other Kotlin Micronaut module in this repo uses `ksp(...)` —
`airbyte-cron`, `airbyte-server`, `airbyte-workload-launcher`. The reporter
appears to be the only one that was not switched.

### Runtime confirmation

On `airbyte/metrics-reporter:2.0.1` (Helm chart V2 2.0.19, `edition: community`),
with `global.metrics.enabled=true`, `global.metrics.otlp.enabled=true` and a valid
`collectorEndpoint`:

- Startup log is 5 substantive lines: Hikari connects, `PushMeterRegistry` starts
(`Publishing metrics for OtlpMeterRegistry every 1m to http://…/v1/metrics`),
Micronaut reports startup complete.
- `EventListeners.startEmitters` logs `registered ${emitters.size} emitters`
unconditionally. **That line never appears**, across multiple clean restarts.
- Management endpoint `/beans` (port 8085) lists **603** bean definitions and
**zero** from `io.airbyte.metrics.reporter` — no `Emitter`, no `EventListeners`,
no `NumPendingJobs`. Only `MetricClient`, which comes from `metrics-lib`.
- `/metrics` returns `{}`.
- The collector never records a single OTLP data point
(`otelcol_receiver_accepted_metric_points` has no `otlp` series at all), while
other receivers on the same collector show millions.
- Connectivity is fine: `POST /v1/metrics` from inside the reporter pod to the
collector returns **HTTP 200 in 1.4 ms**.

`Emitter.emit()` calls `client.count(EST_NUM_METRICS_EMITTED_BY_REPORTER)`
unconditionally, so even with zero rows returned from the database the reporter
would emit something if the beans existed. Nothing is emitted, which is consistent
only with the beans not existing.

## Impact

Any Self-Managed Community deployment on 1.8.0+ that enables `metrics.enabled`
gets an idle JVM and no metrics, with no error to indicate a problem. The failure
is completely silent — the pod is `Running` and `Ready`.

## Fix

`airbyte-metrics/reporter/build.gradle.kts`:

```diff
dependencies {
+ ksp(platform(libs.micronaut.platform))
+ ksp(libs.bundles.micronaut.annotation.processor)

implementation(platform(libs.micronaut.platform))
```

matching `airbyte-cron` / `airbyte-server` / `airbyte-workload-launcher`.

Verification: the built jar should again contain `$Definition` classes and a
`META-INF/micronaut/` directory, and startup should log `registered 11 emitters`.

PR: https://github.com/airbytehq/airbyte-platform/pull/446

## Environment

- Airbyte platform 2.0.1, Helm chart V2 `airbyte-v2/airbyte` 2.0.19
- `global.edition: community`
- k3s single node, external CloudNativePG metadata database
- Reproduced on `airbyte/metrics-reporter` 2.0.0, 2.0.1, and confirmed by jar
inspection on 1.8.0, 2.1.0, 2.1.1, 2.2.0

Guia de contribuição

Abrir o guia de contribuição

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.