PolicyEngine / PolicyEngine/policyengine-household-api

Add first-party calculate latency and uptime analytics

Open
#1,565 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1
Forks
3
Avg merge
6h 34m
Merged PRs (30d)
8

Description

Problem

The developer portal now has a live latency/uptime view, but its latency data is only partial. Today it comes from:

  • the portal synthetic probe, fired every 5 minutes
  • portal-originated Run traffic

That does not measure all real partner production /calculate traffic. The household API already records value-free calculate analytics with trusted client_id, status code, requested/resolved channel, model version, endpoint, method, and variable usage, but it does not store request duration.

Goal

Record first-party calculate request latency in policyengine-household-api so the portal can show true live latency and uptime for production partner API traffic.

Proposed work

  • Add a duration_ms or equivalent integer column to calculate_requests via Alembic migration.
  • Measure wall-clock request duration around the calculate handler in the analytics decorator.
  • Persist duration for successful responses, error responses, and exceptions where analytics is enabled.
  • Expose latency/uptime aggregations from the analytics API, either by extending /analytics/calculate/requests or adding a focused endpoint.
  • Support filters needed by the portal:
    • time window / start and end time
    • resolved channel: current / frontier
    • optionally client_id for staff-only partner drilldown, once authorization rules are decided
  • Return enough aggregate data for the portal to show p50, p95, p99, count, error count, and uptime by time bucket.

Acceptance criteria

  • Real authenticated /calculate requests produce analytics rows with trusted client_id and duration_ms.
  • Failed calculate requests and thrown exceptions record status/error state and duration where possible.
  • Analytics endpoint can return bucketed latency and uptime for current/frontier over recent windows.
  • No request bodies, response bodies, household values, entity IDs, or other sensitive values are logged.
  • Unit tests cover migration/model changes, duration recording, exception paths, and analytics aggregation.
  • Deployed analytics metadata test is updated or extended to assert duration is recorded.

Context

Portal PR: https://github.com/PolicyEngine/policyengine-developer-portal/pull/20

Current limitation in the portal: the dashboard is useful as an availability canary, but it is not yet true production partner latency because household API does not emit first-party latency metrics.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the calculate handler and analytics decorator in policyengine-household-api, then inspect the calculate_requests model and its Alembic migrations. Review /analytics/calculate/requests and the deployed analytics metadata test before deciding how bucketed latency and uptime should be exposed. Done means duration is recorded for successful, failed, and exceptional requests without sensitive values, with aggregation and unit-test coverage for the stated filters and metrics.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
analytics, backend-api-design, databases, observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.