HarperFast / HarperFast/harper

Classify log and status audience: operator vs user (application)

Open
#1,638 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 2h
Merged PRs (30d)
205

Description

## Problem

Harper's log stream (and status surface) mixes two audiences with different responders:

- **Operator/platform** — replication failures, storage errors, resource exhaustion: Harper/Fabric ops must respond.
- **User/application** — errors thrown by customer components, their config mistakes, their upstream failures: the customer must respond.

Today both land in the same stream at the same levels and ship to Datadog identically. Concrete cost: on 2026-07-03 a customer app's error spam (`[releaseStuckRefreshingPages] failed to release ...`) roughly **doubled fleet-wide log ingestion** and stayed there — an app-audience problem billed and paged like a platform one. Conversely, user-relevant problems get buried where customers never see them.

## Proposal

Add an `audience` dimension (`operator` | `user`) to logging and status:

- Loggers created for user components (via the component loader / `forComponent` path for user apps) default to `audience: user`; core subsystem loggers default to `operator`. Explicit override available.
- `logger.status()` entries (PR #372) carry the same field, so `get_status` can present an operator view and a user view.
- Routing policy becomes possible: `operator` warn+ ships to Datadog/paging; `user` messages stay local / go to Studio's notification center / a cheap retention tier, with per-audience level config.
- The Fabric agent config can then filter by audience tag instead of by fragile message patterns.

This is the primary noise-to-signal split for the platform: it separates "what needs *us*" from "what needs *the customer*" at emission time, where the information is known — rather than trying to reconstruct it downstream with log parsing.

## Related
PR #372, structured-log issue (json format carries the field), host-manager agent-pipeline issue.

🤖 Filed by Claude on behalf of Kris.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the component loader's `forComponent` path, the logger creation flow, `logger.status()`, and `get_status`; review related PR #372 and the structured-log issue for existing field conventions. Done means logs and status entries carry the proposed audience values, user-component defaults and core defaults are distinguishable, and explicit overrides and downstream views can be tested.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
backend, observability-sre
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.