pymc-devs / pymc-devs/pymc-examples
Introduction to Bayesian A/B Testing notebook produces SamplingError in version 5.16.2
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 398
- Forks
- 325
- Avg merge
- 9d 15m
- Merged PRs (30d)
- 1
Description
Notebook title:Introduction to Bayesian A/B Testing
Notebook url:https://github.com/pymc-devs/pymc-examples/blob/main/examples/causal_inference/bayesian_ab_testing_introduction.ipynb
Issue description
Running the notebook produces:
A lot of:
pytensor\tensor\elemwise.py:763: RuntimeWarning: invalid value encountered in log
variables = ufunc(*ufunc_args, **ufunc_kwargs)
Then:
Cell In[14], line 1
----> 1 trace_weak, trace_strong = run_scenario_twovariant(
2 variants=["A", "B"],
3 true_rates=[0.23, 0.23],
4 samples_per_variant=100000,
5 weak_prior=BetaPrior(alpha=100, beta=100),
6 strong_prior=BetaPrior(alpha=10000, beta=10000),
7 )
Cell In[13], line 11, in run_scenario_twovariant(variants, true_rates, samples_per_variant, weak_prior, strong_prior)
9 data = [BinomialData(**generated[v].to_dict()) for v in variants]
10 with ConversionModelTwoVariant(priors=weak_prior).create_model(data):
---> 11 trace_weak = pm.sample(draws=5000)
12 with ConversionModelTwoVariant(priors=strong_prior).create_model(data):
13 trace_strong = pm.sample(draws=5000)
File ~\anaconda3\envs\pymc_env\Lib\site-packages\pymc\sampling\mcmc.py:776, in sample(draws, tune, chains, cores, random_seed, progressbar, progressbar_theme, step, var_names, nuts_sampler, initvals, init, jitter_max_retries, n_init, trace, discard_tuned_samples, compute_convergence_checks, keep_warning_stat, return_inferencedata, idata_kwargs, nuts_sampler_kwargs, callback, mp_ctx, blas_cores, model, **kwargs)
774 ip: dict[str, np.ndarray]
775 for ip in initial_points:
--> 776 model.check_start_vals(ip)
777 _check_start_shape(model, ip)
779 if var_names is not None:
File ~\anaconda3\envs\pymc_env\Lib\site-packages\pymc\model\core.py:1793, in Model.check_start_vals(self, start)
1790 initial_eval = self.point_logps(point=elem)
1792 if not all(np.isfinite(v) for v in initial_eval.values()):
-> 1793 raise SamplingError(
1794 "Initial evaluation of model at starting point failed!\n"
1795 f"Starting values:\n{elem}\n\n"
1796 f"Logp initial evaluation results:\n{initial_eval}\n"
1797 "You can call `model.debug()` for more details."
1798 )
SamplingError: Initial evaluation of model at starting point failed!
Starting values:
{'p_logodds__': array([nan, nan])}
Logp initial evaluation results:
{'p': nan, 'y': -inf}
You can call `model.debug()` for more details.
ALSO:
strong/weak_prior name in function run_scenario_twovariant shadows a variabe with a different type in the outer scope. It's probably worth fixing this for readability.
Proposed solution
I don't know how to solve main problem
Suggest changing strong/weak_prior to strong/weak_prior_model in outer scope.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with examples/causal_inference/bayesian_ab_testing_introduction.ipynb and run the failing run_scenario_twovariant cell using PyMC 5.16.2. Investigate the reported invalid log values and SamplingError, and review the strong_prior/weak_prior shadowing noted in the issue. Done means the notebook runs without the reported failure and the variable naming is clear.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100