stan-dev / stan-dev/cmdstan

Optional profiling

Open
#1,137 5 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Feature
Dominant language
C++
Stars
238
Forks
106
Avg merge
2h 29m
Merged PRs (30d)
6

Description

Profiling is a great tool for debugging and performance optimization, but it also adds a bit of overhead to the execution of the Stan program. So it would be nice to skip the profiling statements whenever you don't need them and want the model to run as fast as possible - without the need to manually remove them from the stan program or to always keep two version of the same program (one with and one without profiling statements).

Ideal solution
Ideally, there would be an additonal profile = FALSE/TRUE argument to the sample function of a compiled stan model that would be passed on to CmdStan.

Workaround
As a workaround, one can automatically create a temporary copy of the stan program with all profiling statements removed, compile this temporary copy, and sample from this model without profiling. This requires no changes to CmdStan. The disadvantage is of course that this requires a second compiling of the stan model.

I have implemented this workaround as a wrapper of cmdstan_model in this repo. We have been using it during the development work for epinowcast and it has been quite robust. It also supports included files. It would also be possible to use this workaround for a profile = FALSE/TRUE argument in the sample function.

Context
Regarding the ideal solution of integrating this functionality with CmdStan, I lack the expertise to judge how difficult that would be and probably could not do it myself. If people think a workaround like above is acceptable, I'm happy to help implementing it for cmdstanr, but would need some guidance (and a few details still need to be sorted out).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the compiled model's sample entry point and CmdStan's profiling handling; the issue does not name specific files or tests. Compare the requested profile=FALSE/TRUE behavior with the documented workaround, and verify that sampling can skip profiling without manually editing the Stan program or maintaining a second model version.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
cli, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.