ML-022: Serving Observability
- Dominant language
- Python
- Stars
- 1
- Forks
- 0
- Avg merge
- 6m
- Merged PRs (30d)
- 4
Description
# ML-022 — Serving Observability
## Goal
Add basic observability to the serving layer so that model predictions, latency, errors, and model metadata can be monitored.
## Description
A production model-serving system needs visibility into what model is being served and how the service is behaving.
This story introduces lightweight application-level instrumentation without adding a full monitoring stack.
The implementation should provide structured information that can later be integrated with monitoring systems such as Prometheus, OpenTelemetry, or cloud monitoring.
## Information to Capture
At minimum, capture:
* Model name.
* Model version.
* Model alias.
* Prediction success/failure.
* Prediction latency.
* Request identifier.
* Error information when inference fails.
Example:
```text
request_id=abc123
model=f1-predictor
version=7
alias=champion
latency_ms=18
status=success
```
## Tasks
* [ ] Add request ID generation or propagation.
* [ ] Add prediction latency measurement.
* [ ] Log model name for predictions.
* [ ] Log model version for predictions.
* [ ] Log model alias for predictions.
* [ ] Log successful predictions.
* [ ] Log prediction failures.
* [ ] Log model loading failures.
* [ ] Ensure sensitive request data is not logged unnecessarily.
* [ ] Add structured logging where practical.
* [ ] Add tests for serving instrumentation.
* [ ] Verify instrumentation does not alter prediction behavior.
* [ ] Run MyPy.
* [ ] Run Ruff.
## Acceptance Criteria
* [ ] Every prediction request can be associated with a request ID.
* [ ] Prediction latency is measured.
* [ ] Model metadata is available in serving logs.
* [ ] Successful predictions are observable.
* [ ] Failed predictions are observable.
* [ ] Model-loading failures are observable.
* [ ] Logging does not expose unnecessary sensitive input data.
* [ ] Instrumentation does not change prediction results.
* [ ] Tests cover the instrumentation behavior.
* [ ] MyPy passes without errors.
* [ ] Ruff passes without errors.
## Definition of Done
* [ ] Instrumentation implemented.
* [ ] Logging implemented.
* [ ] Tests added and passing.
* [ ] MyPy passes.
* [ ] Ruff passes.
* [ ] Documentation updated.
* [ ] Changes committed.
Contributor guide
Research direction
Start by locating the serving-layer entry points for prediction and model loading, then trace how request and model metadata currently flow. Add tests for serving instrumentation and verify request IDs, latency, metadata, success/failure logging, and sensitive-data handling without changing prediction behavior; run MyPy and Ruff.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, machine-learning, observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100