joealcorn / joealcorn/laboratory

Run control and candidate in parallel or candidate async

Open
#2 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.3k
Forks
31
PR merge metrics
No merged PRs in 30d

Description

When setting up an experiment, we often want to ensure that the user isn't adversely affected. In order to avoid possible slow downs in the candidate code from affecting the user, would it be possible to run the control and the candidate in parallel? Or better yet, run the candidate completely asynchronously?

I have spent a lot of time writing JavaScript recently so please forgive my lack of ideas on how to solve this problem in Python. Thanks for porting this to Python by the way!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing how an experiment currently runs its control and candidate code, then identify the execution entry points and any existing tests for experiment setup or result collection. Define whether parallel or asynchronous candidate execution is required and how completion, errors, and user impact should be measured before implementation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.