stan-dev / stan-dev/performance-tests-cmdstan
Stan using the same seed=1234 | strange performance tests
Open
@seantalts is already working on this.
Since Apr 14, 2019.
- Dominant language
- Python
- Stars
- 9
- Forks
- 8
- PR merge metrics
- No merged PRs in 30d
Description
method = sample (Default)
sample
num_samples = 1000 (Default)
num_warmup = 1000 (Default)
save_warmup = 0 (Default)
thin = 1 (Default)
adapt
engaged = 1 (Default)
gamma = 0.050000000000000003 (Default)
delta = 0.80000000000000004 (Default)
kappa = 0.75 (Default)
t0 = 10 (Default)
init_buffer = 75 (Default)
term_buffer = 50 (Default)
window = 25 (Default)
algorithm = hmc (Default)
hmc
engine = nuts (Default)
nuts
max_depth = 10 (Default)
metric = diag_e (Default)
metric_file = (Default)
stepsize = 1 (Default)
stepsize_jitter = 0 (Default)
id = 0 (Default)
data
file = XXX
init = 2 (Default)
random
seed = 1234
output
file = XXX
diagnostic_file = (Default)
refresh = 100 (Default)
Tests I found during Jenkins runs that give strange performance reports on the same machine in subsequent runs.
PrevBuildURI90% | CurrentBuildURI90% | RelativeDiff | RelativeDiff(%)
286 40526 40240 14069.93%
example-models/bugs_examples/vol1/pump/pump.data.R
---
443 42348 41905 9459.37%
example-models.bugs_examples.vol1.dyes.dyes
---
23237 533 -22704 -97.71%
example-models.bugs_examples.vol1.equiv.equiv
---
8302 39174 30872 371.86%
example-models.bugs_examples.vol1.leuk.leuk
---
7807 2274 -5533 -70.87%
example-models.bugs_examples.vol1.mice.mice
---
163 365 202 123.93%
example-models.bugs_examples.vol2.beetles.beetles_cloglog
---
7807 39129 31322 401.20%
example-models.bugs_examples.vol1.mice.mice
---
23237 506 -22731 -97.82%
example-models.bugs_examples.vol1.equiv.equiv
---
153 114870 114717 74978.43%
example-models.bugs_examples.vol2.beetles.beetles_probit
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.