apache / apache/maven

Build output overhaul: structured build report, diagnostic collector, console modes

Open
#12,571 0 comments 0 reactions 0 assignees View on GitHub
enhancement mvn4
Dominant language
Java
Stars
5.3k
Forks
3.1k
Avg merge
20h 42m
Merged PRs (30d)
297

Description

## Summary

Maven's build output needs a holistic redesign for 4.1.0. Today's output has fundamental problems that affect humans, CI systems, and LLM-based tools equally:

1. **No signal hierarchy** — plugin execution banners, download progress, enforcer rule passes, and actual compilation errors are all `[INFO]`. Consumers cannot distinguish noise from signal without parsing content.
2. **Parallel builds produce interleaved garbage** — with `-T`, output from concurrent modules interleaves unpredictably, making parallel builds unreadable.
3. **Warnings are fire-and-forget** — a deprecation warning in module 3/280 is gone by the time the build finishes. No deduplication, no summary. In a real test on Apache Maven itself, **154 out of 257 filtered lines were duplicate deprecation warnings** — same message, different file coordinates.
4. **No persistent structured data** — everything goes to the console or nowhere. If you want to re-read build results (timing, warnings, test failures), you must re-run the build. There is no structured artifact to query.
5. **No machine-readable output** — tools like [rtk](https://github.com/rtk-ai/rtk), IDE integrations, and LLM agents must regex-parse Maven's text output to extract structured information.

This issue proposes a layered architecture that addresses all five problems through four implementation phases, each independently shippable and useful. It supersedes and unifies #11819, #8571 (MNG-6662), #8976 (MNG-7484), and relates to #10872 (MNG-8071), #11088, #8973 (MNG-7941).

## Architecture

The core insight: **the build should produce a structured report as a first-class artifact. Console output, log files, and machine-readable formats are all renderers of the same underlying data — not the data itself.**

```
┌─────────────────────────────────────────────────────────────────┐
│ Build Execution │
│ (lifecycle events, mojo execution, downloads, test results) │
└──────────────────────────┬──────────────────────────────────────┘


┌─────────────────────────────────────────────────────────────────┐
│ Layer 1: Build Report (data model) │
│ │
│ BuildReport │
│ ├── metadata (timestamp, goals, profiles, maven/java version) │
│ ├── timing (wall clock, per-phase breakdown) │
│ ├── modules[] │
│ │ ├── gav, status, duration │
│ │ ├── phases[] → mojos[] (plugin:goal, duration, status) │
│ │ ├── compilations[] (sources, warnings, errors) │
│ │ ├── testResults (run/failed/errors/skipped, duration) │
│ │ └── downloads[] (artifact, size, duration, cached?) │
│ ├── diagnostics[] (key, severity, message, locations[], │
│ │ suggestion, documentationUrl, count) │
│ ├── failures[] (module, phase, mojo, error, stripped trace) │
│ └── reactorSummary[] │
│ │
│ Persisted to: target/build-report.json │
└──────────────────────────┬──────────────────────────────────────┘


┌─────────────────────────────────────────────────────────────────┐
│ Layer 2: Collection Services (Maven 4 API) │
│ │
│ DiagnosticCollector (public API — plugins use this) │
│ BuildReportCollector (internal — filled from lifecycle events) │
└──────────────────────────┬──────────────────────────────────────┘

┌────────────┼────────────┬──────────────┐
▼ ▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌────────────────┐
│ Smart │ │ CI/Batch │ │ Verbose │ │ Machine │
│ Terminal │ │ Output │ │ (today) │ │ (JSON lines) │
│ (JLine) │ │ (plain) │ │ │ │ │
└──────────┘ └──────────┘ └──────────┘ └────────────────┘
Layer 3: Renderers (selected via --console flag)
```

## Layer 1: Build Report (data model + persistence)

### What

A structured `BuildReport` written to `target/build-report.json` at the end of every build.

### Why persisted

- **Re-read after build** — LLMs, developers, and CI tools can query results without rebuilding. If an LLM agent ran the build, it can read the structured report on the next turn without re-running.
- **Diff two reports** — "why is this build 30s slower?" becomes a JSON diff.
- **Feed into analytics** — CI dashboards, Develocity-style trend analysis, or local tooling.
- **Extensions consume it** — projects like [maven-build-scanner](https://github.com/intuit/maven-build-scanner) (158⭐) and [maven-buildtime-extension](https://github.com/timgifford/maven-buildtime-extension) (134⭐) currently use EventSpy to collect this same data independently. A standard report format lets them consume rather than re-collect.
- **Re-render** — `mvn build-report:show` can re-display the last build's summary without rebuilding.

### Data model sketch

```java
public interface BuildReport {
BuildMetadata metadata();
Duration totalDuration();
List modules();
List diagnostics(); // deduplicated
List failures();
ReactorSummary reactorSummary();
BuildStatus status(); // SUCCESS, FAILURE
}

public interface ModuleReport {
Project project();
BuildStatus status();
Duration duration();
List phases();
@Nullable TestReport testResults();
List compilations();
List downloads();
}
```

## Layer 2: Collection Services (Maven 4 API)

### 2a. `DiagnosticCollector` — the warning collector

This is a new service in `maven-api-core`, retrievable via `session.getService(DiagnosticCollector.class)`.

```java
public interface Diagnostic {
enum Severity { INFO, WARNING, ERROR }

@Nonnull String key(); // dedup key, e.g. "deprecated-source-target"
@Nonnull Severity severity();
@Nonnull String message();
@Nullable String source(); // plugin GAV or module
@Nullable String file();
int line();
int column();
@Nullable String suggestion(); // actionable fix text
@Nullable String documentationUrl();
}

public interface DiagnosticCollector extends Service {
void report(@Nonnull Diagnostic diagnostic);
@Nonnull List getDiagnostics();
@Nonnull List getSummary(); // deduplicated, with counts
boolean hasWarnings();
boolean hasErrors();
}
```

**Design notes:**
- Follows the existing `BuilderProblem` / `ProblemCollector` pattern already in the Maven 4 API, but scoped to the entire build lifecycle rather than just model building.
- Thread-safe, session-scoped (critical for parallel builds).
- The `key` field enables deduplication across modules and suppression via `-Dmaven.diagnostic.suppress=deprecated-source-target`.

### 2b. Backward compatibility with `Log.warn()`

Maven 3 plugins use `getLog().warn(message)` — fire-and-forget. To give these plugins partial benefits without code changes:

- Intercept `Log.warn()` calls and feed them into `DiagnosticCollector` with auto-generated keys (hash of message text with file coordinates stripped for dedup).
- Old plugins get: deduplication and summary at end of build.
- Old plugins don't get: suppressibility by key, suggestions, documentation URLs.

### 2c. Migration incentive

This creates a natural **carrot for migrating plugins to the Maven 4 API**:

| Feature | Maven 3 plugin (`Log.warn()`) | Maven 4 plugin (`DiagnosticCollector`) |
|---|---|---|
| Warning shown inline | ✅ | ✅ (controllable) |
| Deduplicated | ✅ (auto, by message hash) | ✅ (by explicit key — more precise) |
| Summary at end of build | ✅ (best-effort) | ✅ (structured, with counts) |
| Actionable suggestion | ❌ | ✅ |
| Documentation link | ❌ | ✅ |
| Machine-readable (JSON) | ❌ | ✅ |
| Per-module grouping | ❌ | ✅ |
| Suppressible by key | ❌ | ✅ |

**High-impact plugins to migrate first:**
1. `maven-compiler-plugin` — deprecation/unchecked warnings (the #1 source of duplicate noise)
2. `maven-enforcer-plugin` — rule pass/fail results
3. `maven-surefire-plugin` — test result summaries, flaky test warnings
4. `maven-dependency-plugin` — unused/undeclared dependency warnings

## Layer 3: Renderers (console output modes)

All renderers consume the same `BuildReportCollector` data. The console is no longer "whatever SLF4J prints" — it's a structured render of collected events.

### 3a. Smart Terminal (interactive TTY) — `--console=rich`

Using JLine `Display` API (already a Maven dependency, currently unused for status display):

```
✓ my-core 1.2s
✓ my-api 0.8s
Building my-app 2.0.0 [3/12]
✓ clean → resources → compile 1.6s
● surefire:test 2.1s
↓ junit-jupiter-api-5.10.jar 234/512 KB

────────────────────────────────────────────────────────────────
my-core ✓ my-api ✓ my-app ● my-web ⏳ [4/12] 12s
```

- Fixed status bar at bottom — currently building modules, reactor position, elapsed time
- Parallel-aware — each concurrent module gets a row (proven by mvnd's `TerminalOutput.java`)
- Downloads shown inline with progress, then cleared
- Warnings/errors bubble up above the status area
- Plugin `[INFO]` suppressed from console → written to build report + log file

### 3b. CI / Batch mode — `--console=plain`

```
[INFO] Building my-core 2.0.0 [1/12] .................. SUCCESS [1.2s]
[INFO] Building my-api 2.0.0 [2/12] ................... SUCCESS [0.8s]
[WARN] my-utils: source/target 8 is deprecated
[INFO] Building my-app 2.0.0 [4/12] ................... FAILURE [3.1s]

── Warnings (2 unique, 23 occurrences) ────────────────────────
⚠ source/target 8 is deprecated, use 11+ (×23)
→ Update to 11 or higher
in: my-core, my-api, my-utils, +18 more modules

── Failure ────────────────────────────────────────────────────
my-app > maven-compiler-plugin:compile
src/main/java/App.java:42: cannot find symbol: class Foo

BUILD FAILURE 12 modules | 11 passed | 1 failed | 32.1s
Full report: target/build-report.json
```

This is optimal for both CI logs and LLM agents — compact, actionable, no noise, and a pointer to the full structured report.

### 3c. Verbose — `--console=verbose`

Full mojo-level output like today's default, plus the structured summary appended at the end. This becomes opt-in rather than the default.

### 3d. Debug — `-X` / `--debug`

Unchanged — full debug trace for troubleshooting Maven itself.

### 3e. Machine — `--console=machine`

JSON lines to stdout — one event per line, typed:

```json
{"event":"module.started","module":"my-core","index":1,"total":12,"timestamp":"..."}
{"event":"mojo.succeeded","module":"my-core","plugin":"compiler","goal":"compile","duration":1.2}
{"event":"diagnostic","key":"deprecated-source-target","severity":"WARNING","count":3}
{"event":"test.results","module":"my-app","run":58,"failed":2,"errors":0,"skipped":0}
{"event":"build.finished","status":"FAILURE","duration":32.1}
```

## Layer 4: Console mode selection

### `--console` flag

| Flag | Behavior |
|---|---|
| `--console=auto` (default) | TTY → rich; pipe/CI → plain |
| `--console=rich` | Force smart terminal |
| `--console=plain` | CI/batch mode (no ANSI, one line per module) |
| `--console=verbose` | Full mojo output (today's behavior) |
| `--console=machine` | JSON lines |

### `--warning-mode` flag

| Flag | Behavior |
|---|---|
| `--warning-mode=summary` (default) | Collect + dedupe, show at end |
| `--warning-mode=all` | Inline + summary |
| `--warning-mode=none` | Suppress |
| `--warning-mode=fail` | Treat warnings as build errors |

### CI auto-detection

When `CIDetector` identifies a CI environment (GitHub Actions, Jenkins, Travis, CircleCI, TeamCity), automatically apply:
- `--console=plain` (unless overridden)
- `--batch-mode` (no interactive prompts)
- `--show-errors` (stack traces on failure)

Per #11088 and PR #11104.

## Implementation Phases

### Phase 0: Build Report foundation (no visible output changes)

- Define `BuildReport` data model in `maven-api-core`
- Implement `BuildReportCollector` (internal), populated from `BuildEventListener` events
- Write `target/build-report.json` at session end
- Immediately useful: tools/LLMs read structured data, extensions consume it

### Phase 1: DiagnosticCollector + warning summary (first visible improvement)

- `Diagnostic` + `DiagnosticCollector` API in `maven-api-core`
- `DefaultDiagnosticCollector` implementation — thread-safe, session-scoped
- Intercept `Log.warn()` for Maven 3 plugins (auto-key by message hash)
- Print deduplicated warning summary after reactor summary in `ExecutionEventLogger.sessionEnded()`
- Add `--warning-mode` flag
- Start migrating `maven-compiler-plugin` to use structured diagnostics

### Phase 2: CI mode + batch output improvements

- Merge CI auto-detection (#11088 / PR #11104)
- Implement `--console=plain` — one-line-per-module progress, structured error/warning summary
- Merge transfer logging improvements (PR #1238) — `summary` mode as default
- Suppress plugin `[INFO]` from console by default, write to build report
- Add `--console` flag with `auto`, `plain`, `verbose`

### Phase 3: Smart terminal

- JLine `Display`-based status area (fixed bottom rows)
- Per-module progress rows (parallel-aware)
- Download progress inline, cleared when done
- `--console=rich` mode
- Port proven patterns from mvnd's `TerminalOutput.java`
- Prerequisite: JLine 4.x upgrade (#11028) for better JPMS support

### Phase 4: Machine-readable output + plugin ecosystem migration

- `--console=machine` (JSON lines)
- Plugin migration guide for adopting `DiagnosticCollector`
- Migrate remaining core plugins (enforcer, surefire, dependency)
- `mvn build-report:show` goal to re-render last build's summary

## Related Issues

- #11819 — Redesign Maven console output (superseded by this issue)
- #8571 (MNG-6662) — More concise logging
- #8976 (MNG-7484) — Warning summary at end of build
- #10872 (MNG-8071) — Build in parallel by default
- #11088 — CI env variable defaults
- #8973 (MNG-7941) — -ntp vs --batch-mode
- #11028 — Upgrade to JLine 4.x
- PR #1238 — New Maven4 transfer logging
- PR #11104 — CI optimizations

## Prior Art

| Tool/Project | What it does | What we learn |
|---|---|---|
| **Gradle** | `--console=rich\|plain\|auto`, `--warning-mode=summary\|all\|fail`, `LIFECYCLE` log level | Console mode selection, warning control flags |
| **Bazel** | Build Event Protocol (JSON), curses-based last-N-lines status area | Structured event persistence, terminal status UI |
| **Cargo** | TTY-aware progress bar, `--message-format=json`, automatic warning dedup | Machine-readable output, smart TTY detection |
| **mvnd** | JLine `TerminalOutput` with per-module progress rows | Parallel-aware terminal rendering (proven impl) |
| **takari/concurrent-build-logger** | SLF4J MDC per-project tagging, `byproject` buffered output, per-project `build.log` | Parallel output grouping via Logback |
| **Intuit/maven-build-scanner** | EventSpy collecting timing + rendering HTML reports | Build report data model, analytics use case |
| **rtk** | External Maven output filter — regex-based noise reduction | Proves the problem exists; a native solution eliminates the need |

## Target

Maven 4.1.0

Contributor guide

Open the contributing guide

Research direction

Start by reading the existing BuilderProblem and ProblemCollector patterns in the Maven 4 API, then inspect maven-api-core and the lifecycle event handling around BuildReportCollector. The issue spans DiagnosticCollector, report persistence, and the --console renderers, but names no implementation files or tests. Done would require an agreed, independently shippable phase with corresponding behavior and validation.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system, cli, developer-experience
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.