stan-dev / stan-dev/rstanarm

Warmup time is sensitive to the scale of data

Open
#281 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
R
Stars
401
Forks
136
PR merge metrics
No merged PRs in 30d

Description

Summary:

In a simple linear Gaussian example with p= and n=, the scale of data can affect the warmup time from around 1 second to more than 100 seconds. This is likely to be related to initial values being very far from the typical set. Since stan_glm is restricted to certain models, it could be possible to scale the initial values based on the data scale.

Reproducible Steps:

Data in http://www.stat.columbia.edu/~gelman/regression/

earnings_all <- read.csv("Earnings/data","earnings.csv")
earnings_all$positive <- earnings_all$earn > 0
# only non-zero earnings
earnings <- earnings_all[earnings_all$positive, ]
> sd(earnings$earn)
[1] 19497.08
> M_1 <- stan_glm(earn ~ height + male, data = earnings, seed=987, refresh=0)
 Elapsed Time: 8.79859 seconds (Warm-up)
               1.16896 seconds (Sampling)
               9.96755 seconds (Total)
 Elapsed Time: 10.3458 seconds (Warm-up)
               0.952707 seconds (Sampling)
               11.2985 seconds (Total)
 Elapsed Time: 15.4682 seconds (Warm-up)
               0.915143 seconds (Sampling)
               16.3834 seconds (Total)
 Elapsed Time: 16.1475 seconds (Warm-up)
               0.875306 seconds (Sampling)
               17.0228 seconds (Total)
> M_1 <- stan_glm(earn*10000 ~ height + male, data = earnings, seed=987, refresh=0)
 Elapsed Time: 100.496 seconds (Warm-up)
               1.36169 seconds (Sampling)
               101.858 seconds (Total)
 Elapsed Time: 110.283 seconds (Warm-up)
               1.46981 seconds (Sampling)
               111.753 seconds (Total)
 Elapsed Time: 110.861 seconds (Warm-up)
               1.36417 seconds (Sampling)
               112.226 seconds (Total)
 Elapsed Time: 115.265 seconds (Warm-up)
               0.782603 seconds (Sampling)
               116.047 seconds (Total)
> M_1 <- stan_glm(earn/10000 ~ height + male, data = earnings, seed=987, refresh=0)
 Elapsed Time: 0.669384 seconds (Warm-up)
               0.941708 seconds (Sampling)
               1.61109 seconds (Total)
 Elapsed Time: 0.859793 seconds (Warm-up)
               1.00288 seconds (Sampling)
               1.86267 seconds (Total)
 Elapsed Time: 1.11034 seconds (Warm-up)
               1.02959 seconds (Sampling)
               2.13993 seconds (Total)
 Elapsed Time: 0.98462 seconds (Warm-up)
               0.959394 seconds (Sampling)
               1.94401 seconds (Total)
RStanARM Version:

The version of the rstanarm package you are running (e.g., from packageVersion("rstanarm"))
[1] ‘2.17.4’

R Version:

The version of R you are running (e.g., from getRversion())
[1] "R version 3.4.4 (2018-03-15)"

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the supplied R reproduction with stan_glm and compare warmup times for the original, multiplied, and divided response scales. No source file or test is named; done would require demonstrating that warmup is no longer strongly affected by the data scale and adding appropriate coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.