PolicyEngine / PolicyEngine/microcosm

Logbook: record requested and realized exact-k cardinality

Open
#641 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

Summary

PR #632 added Logbook as the append-only trace of build attempts, including sparse and dense builds. However, Logbook rows do not currently register the exact-k cardinality associated with a build.

The current build-row contract records identity, pipeline, rung, seed, phases, disposition, and artifact location, but has no typed field for either the requested k or the realized output record count. Consequently, consumers of builds_public cannot reliably filter or compare exact-k builds without parsing naming conventions or private/free-form receipt data.

Why this matters

Exact-k is a defining property of sparse outputs. It should be queryable as build metadata rather than inferred from a release ID such as ...-k57240-....

Without first-class cardinality metadata, Logbook cannot yet fully support the use case described in #632—seeing and comparing “all sparses, all denses”—because it cannot directly answer:

  • which builds requested a particular k;
  • whether the realized output actually contained exactly k records;
  • which rows represent full (k=N) versus sparse outputs; or
  • how fit, cost, gates, and disposition vary by k.

This is also a prerequisite for the dataset-family discovery work proposed in #637.

Proposed contract

Add typed, publicly queryable cardinality metadata for applicable build rows. At minimum:

  • requested_k: the exact support cardinality requested by the build;
  • realized_k: the validated record count in the packaged output; and
  • record_unit: the entity being counted, initially household, so that k is not ambiguous across future pipelines.

The exact storage design is open. These may be versioned build-row fields or an insert-only Logbook extension keyed to the authenticated build row. In either case:

  • do not rely on parsing build_id, release tags, or artifact_location;
  • do not bury k only inside gate_verdicts, which is intentionally excluded from builds_public;
  • expose the fields through the public read model;
  • include them in authenticated provenance for new rows; and
  • preserve verification of the existing hash chain and its backfilled rows through an explicit schema/digest-versioning or compatible extension strategy.

Pipeline integration

Wire the exact-k build/release path to Logbook so a terminal exact-k attempt records its cardinality alongside its artifact and disposition. Exact-k successful builds should assert and record requested_k == realized_k before they may be marked published or certified.

For non-exact-k builds, failures before cardinality is known, and historical rows that cannot be verified, the new values may remain null. Historical values should only be backfilled when an immutable release manifest or artifact provides sufficient evidence.

Acceptance criteria

  • A new exact-k Logbook entry records requested k, realized k, and the counted entity.
  • Published/certified exact-k rows expose this metadata through the public Logbook view.
  • Consumers can query and order builds by k without parsing identifiers or free-form JSON.
  • A successful exact-k publication fails closed if realized cardinality differs from requested k.
  • Existing chained rows continue to validate unchanged.
  • The exact-k launcher/publication workflow emits the corresponding Logbook record.
  • Tests cover new rows, nullable/non-applicable rows, public projection, and backward compatibility.

Related: #637.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the Logbook build-row contract, the builds_public read model, and the exact-k launcher/publication workflow named in the issue; no file or test paths are provided. Trace how new rows, authenticated provenance, and chained-row validation are handled. Done means exact-k rows expose requested_k, realized_k, and record_unit publicly, enforce equality before publication, and have coverage for nullability and backward compatibility.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.