Emil57 / Emil57/ml-platform

ML-022: Serving Observability

Open
#40 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1
Forks
0
Avg merge
6m
Merged PRs (30d)
4

Description

# ML-022 — Serving Observability

## Goal

Add basic observability to the serving layer so that model predictions, latency, errors, and model metadata can be monitored.

## Description

A production model-serving system needs visibility into what model is being served and how the service is behaving.

This story introduces lightweight application-level instrumentation without adding a full monitoring stack.

The implementation should provide structured information that can later be integrated with monitoring systems such as Prometheus, OpenTelemetry, or cloud monitoring.

## Information to Capture

At minimum, capture:

* Model name.
* Model version.
* Model alias.
* Prediction success/failure.
* Prediction latency.
* Request identifier.
* Error information when inference fails.

Example:

```text
request_id=abc123
model=f1-predictor
version=7
alias=champion
latency_ms=18
status=success
```

## Tasks

* [ ] Add request ID generation or propagation.
* [ ] Add prediction latency measurement.
* [ ] Log model name for predictions.
* [ ] Log model version for predictions.
* [ ] Log model alias for predictions.
* [ ] Log successful predictions.
* [ ] Log prediction failures.
* [ ] Log model loading failures.
* [ ] Ensure sensitive request data is not logged unnecessarily.
* [ ] Add structured logging where practical.
* [ ] Add tests for serving instrumentation.
* [ ] Verify instrumentation does not alter prediction behavior.
* [ ] Run MyPy.
* [ ] Run Ruff.

## Acceptance Criteria

* [ ] Every prediction request can be associated with a request ID.
* [ ] Prediction latency is measured.
* [ ] Model metadata is available in serving logs.
* [ ] Successful predictions are observable.
* [ ] Failed predictions are observable.
* [ ] Model-loading failures are observable.
* [ ] Logging does not expose unnecessary sensitive input data.
* [ ] Instrumentation does not change prediction results.
* [ ] Tests cover the instrumentation behavior.
* [ ] MyPy passes without errors.
* [ ] Ruff passes without errors.

## Definition of Done

* [ ] Instrumentation implemented.
* [ ] Logging implemented.
* [ ] Tests added and passing.
* [ ] MyPy passes.
* [ ] Ruff passes.
* [ ] Documentation updated.
* [ ] Changes committed.

Contributor guide

Open the contributing guide

Research direction

Start by locating the serving-layer entry points for prediction and model loading, then trace how request and model metadata currently flow. Add tests for serving instrumentation and verify request IDs, latency, metadata, success/failure logging, and sensitive-data handling without changing prediction behavior; run MyPy and Ruff.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, machine-learning, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.