godofecht / godofecht/flow-scikit

[Bench #478] Benchmark every Flow-implemented estimator operation, not only the 19 headline rows

Open
#497 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2
Forks
0
Avg merge
4h 33m
Merged PRs (30d)
9

Description

Parent: #478

The architecture inventory contains 491 sklearn estimator-operation rows, while the canonical headline intentionally covers only 19 estimator/dataset combinations. Any Flow-implemented estimator or operation outside that set is currently capable of hiding a severe regression.

Build an implementation-surface registry and require every public Flow estimator operation to have at least correctness/parity smoke, representative warm timing, scaled timing classification, memory evidence tier and execution-substrate attribution. Unsupported sklearn operations should be distinguished from implemented-but-unbenchmarked operations.

Validation is local-only. Do not add, enable, trigger, inspect, wait for, or rely on GitHub Actions or other hosted CI/workflows. Coverage ratcheting should be enforced by a deterministic local command from a clean checkout and recorded in machine-readable artifacts keyed to the exact source SHA/environment.

Acceptance: implemented public operations with no benchmark evidence = 0; coverage is machine-readable and locally ratcheted; the 19-row suite remains the stable headline subset, while the full surface produces a broader wins/ties/losses dashboard and new losses become focused issues.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are named. Start by inspecting the benchmark architecture associated with parent issue #478 and map the stated 19-row headline subset against the full estimator-operation inventory. Done means the acceptance criteria are met through a deterministic local command and machine-readable coverage and dashboard artifacts.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, scikit-learn
Domain
machine-learning, performance, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.