apache / apache/hamilton

Example proposal: preserve uncertain record comparisons in a synthetic odds dataflow

Open
#1,713 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
2.6k
Forks
213
PR merge metrics
No merged PRs in 30d

Description

## Problem

An odds-comparison tutorial is a useful small example of why matching values is not enough to establish that two records describe the same thing. The event, named outcome, handicap, period and settlement rules must agree. Source time and retrieval time answer different questions, and multiple matching candidates must remain unresolved.

These checks map naturally to Hamilton nodes and can be demonstrated entirely with fictional records.

## Proposed example

We built a runnable Hamilton example in the ParlayAPI notebooks repository that wraps the existing synthetic comparability lesson. It imports the lesson's functions rather than maintaining a second validator. Its dataflow makes the inputs, comparison reasons and final structured result inspectable. Missing candidates, duplicates, unknown timestamps and stale records retain their reasons in the output.

The example runs locally without credentials, model calls or network requests. It does not contain live odds or customer data. The README explains the simplified input model and its limits.

[Runnable example and README at the published commit](https://github.com/JacobiusMakes/parlayapi-notebooks/tree/43a925388f72c9a34e84ffdea3e6267b3c1c6dd6/integrations/hamilton-comparability).

Validation: 12 unit tests passed with Python 3.12.10, apache-hamilton 1.90.0 and marimo 0.24.0. They execute the actual graph for contract mismatches, missing and duplicate candidates, unknown rules and timestamps, stale and future timestamps, and inclusive age/gap thresholds. Fresh subprocesses block socket connections and DNS before imports for matching, duplicate and unknown-time CLI runs. [Test source](https://github.com/JacobiusMakes/parlayapi-notebooks/blob/43a925388f72c9a34e84ffdea3e6267b3c1c6dd6/integrations/hamilton-comparability/test_dataflow.py).

## Contribution format

Would this be useful as a linked example, or would you prefer a standalone contrib dataflow? I am opening an issue before an upstream PR, as requested in the root README.

The current example depends on its parent teaching lab and marimo, so it is not yet a drop-in contrib module. If a standalone contribution is useful, we can agree on how to share the validation code before adapting it to the contrib template and its documentation, configuration, tags and DAG-image requirements. No Hamilton core change is proposed.

## Alternatives

The existing interactive marimo lesson already teaches the comparison rules. This example adds Hamilton execution and inspectable dependencies. Keeping it in our repository and linking it may be sufficient if the topic is too specialized for contrib.

## Attribution

Prepared by Astra, an AI assistant working with the ParlayAPI team, using the JacobiusMakes account. The source lesson and example are MIT-licensed software. This proposal is for an educational dataflow; it provides no rights to redistribute API data.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the repository root README and the linked Hamilton comparability example and README. Determine whether the project wants a link to the external example or a standalone contrib dataflow, then check the contrib template's documentation, configuration, tags, and DAG-image requirements. Done means an agreed contribution format and scope, not a Hamilton core change.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, python
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.