instadeepai / instadeepai/winnow

refactor: end calibration-feature pickle support

Open
#243 0 comments 0 reactions 0 assignees View on GitHub
refactor
Dominant language
Python
Stars
11
Forks
2
Avg merge
1d 8h
Merged PRs (30d)
1

Description

## Summary

Winnow v2.0.0 replaced `calibrator.pkl` checkpoints with `config.json` and `model.safetensors`. `ProbabilityCalibrator.load()` already rejects legacy pickle checkpoints clearly, but `RetentionTimeFeature` still supports direct pickling and backfills older state.

Remove this remaining feature-pickle compatibility so v2 has one explicit model persistence format.

This coincides with work in [PR #239: Allow targeting a self-hosted Koina server](https://github.com/instadeepai/winnow/pull/239).

## Change

- Explicitly reject pickling and unpickling `CalibrationFeatures` through the common base class and direct users to `ProbabilityCalibrator.save()` / `load()`.
- Remove `RetentionTimeFeature.__getstate__`, `__setstate__`, and its legacy `min_train_points` backfill.
- Replace feature pickle round-trip tests with tests for clear unsupported format errors.
- Confirm current JSON+safetensors checkpoints still round-trip.
- Add a short migration/release note.

## Acceptance criteria

- Calibrator and feature objects are never loaded from pickle.
- Feature pickle attempts fail immediately with an actionable error.
- Legacy `calibrator.pkl` directories retain their existing migration error.
- JSON+safetensors checkpoint tests pass.
- Prediction-data pickle support is unaffected.

Contributor guide

Open the contributing guide

Research direction

Start with ProbabilityCalibrator.load(), the common base class for CalibrationFeatures, and RetentionTimeFeature.__getstate__ and __setstate__. Inspect the existing feature pickle round-trip tests and current JSON+safetensors checkpoint tests. Done means feature pickle operations fail with actionable errors, legacy calibrator.pkl migration errors remain, current checkpoints round-trip, and prediction-data pickle support is unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.