Warmup time is sensitive to the scale of data
Nobody has claimed this yet.
- Dominant language
- R
- Stars
- 401
- Forks
- 136
- PR merge metrics
- No merged PRs in 30d
Description
Summary:
In a simple linear Gaussian example with p= and n=, the scale of data can affect the warmup time from around 1 second to more than 100 seconds. This is likely to be related to initial values being very far from the typical set. Since stan_glm is restricted to certain models, it could be possible to scale the initial values based on the data scale.
Reproducible Steps:
Data in http://www.stat.columbia.edu/~gelman/regression/
earnings_all <- read.csv("Earnings/data","earnings.csv")
earnings_all$positive <- earnings_all$earn > 0
# only non-zero earnings
earnings <- earnings_all[earnings_all$positive, ]
> sd(earnings$earn)
[1] 19497.08
> M_1 <- stan_glm(earn ~ height + male, data = earnings, seed=987, refresh=0)
Elapsed Time: 8.79859 seconds (Warm-up)
1.16896 seconds (Sampling)
9.96755 seconds (Total)
Elapsed Time: 10.3458 seconds (Warm-up)
0.952707 seconds (Sampling)
11.2985 seconds (Total)
Elapsed Time: 15.4682 seconds (Warm-up)
0.915143 seconds (Sampling)
16.3834 seconds (Total)
Elapsed Time: 16.1475 seconds (Warm-up)
0.875306 seconds (Sampling)
17.0228 seconds (Total)
> M_1 <- stan_glm(earn*10000 ~ height + male, data = earnings, seed=987, refresh=0)
Elapsed Time: 100.496 seconds (Warm-up)
1.36169 seconds (Sampling)
101.858 seconds (Total)
Elapsed Time: 110.283 seconds (Warm-up)
1.46981 seconds (Sampling)
111.753 seconds (Total)
Elapsed Time: 110.861 seconds (Warm-up)
1.36417 seconds (Sampling)
112.226 seconds (Total)
Elapsed Time: 115.265 seconds (Warm-up)
0.782603 seconds (Sampling)
116.047 seconds (Total)
> M_1 <- stan_glm(earn/10000 ~ height + male, data = earnings, seed=987, refresh=0)
Elapsed Time: 0.669384 seconds (Warm-up)
0.941708 seconds (Sampling)
1.61109 seconds (Total)
Elapsed Time: 0.859793 seconds (Warm-up)
1.00288 seconds (Sampling)
1.86267 seconds (Total)
Elapsed Time: 1.11034 seconds (Warm-up)
1.02959 seconds (Sampling)
2.13993 seconds (Total)
Elapsed Time: 0.98462 seconds (Warm-up)
0.959394 seconds (Sampling)
1.94401 seconds (Total)
RStanARM Version:
The version of the rstanarm package you are running (e.g., from packageVersion("rstanarm"))
[1] ‘2.17.4’
R Version:
The version of R you are running (e.g., from getRversion())
[1] "R version 3.4.4 (2018-03-15)"
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the supplied R reproduction with stan_glm and compare warmup times for the original, multiplied, and divided response scales. No source file or test is named; done would require demonstrating that warmup is no longer strongly affected by the data scale and adding appropriate coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100