resgroup / resgroup/wind-up

Uplift plots are not based on the final complete analysis result

Open
#78 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
29
Forks
11
Avg merge
1d 9h
Merged PRs (30d)
11

Description

Issue Summary

The uplift plots currently saved to the plots folder show only the preliminary 'forward' test-reference analysis results and do not reflect the final reported uplift values.

Current Behaviour

Uplift plots are generated from the initial 'forward' test-reference analysis
These plots do not incorporate the full analysis pipeline including:

  • Bootstrapping calculations
  • Test-reference reversal analysis

The plots therefore show different (higher) uplift values than the final reported results

Expected Behaviour

The uplift plots in the plots folder should be based on the final uplift results that have been processed through the complete analysis pipeline, including bootstrapping and test-reference reversal calculations.

Additional Context

Final uplift plots will typically show the same or lower uplift values compared to the current 'forward'-only plots due to the uncertainty methodology
This change will ensure consistency between the visual outputs and the reported final results.

A couple of example uplift plots that are currently based on the 'forward' analysis only

Image Image

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are named. Trace where uplift plots are generated and how the complete analysis produces final uplift results, including bootstrapping and test-reference reversal; done means plots in the plots folder use those final values rather than the preliminary forward analysis.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-visualization
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.