stan-dev / stan-dev/cmdstanr

Compiled model methods and `expose_functions` fail to load when RcppParallel's TBB is already in the session

Open Beginner friendly
#1,270 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
R
Stars
160
Forks
69
Avg merge
1d 19h
Merged PRs (30d)
15

Description

I noticed this while working on v1.0 and also when reviewing #1274. I asked Claude to write up the report below.


init_model_methods() and expose_functions() compile the model methods with Rcpp and loads the result. If RcppParallel's libtbb.dylib is already loaded in the R process, which happens as soon as rstan or brms is loaded, the load fails:

Error in dyn.load(".../sourceCpp_2.so") :
  unable to load shared object '.../sourceCpp_2.so':
  dlopen(...): Symbol not found: __ZN3tbb8internal23allocate_via_handler_v3Em
  Expected in: /Users/.../R/x86_64/4.5/library/RcppParallel/lib/libtbb.dylib

Without RcppParallel in the session the same call succeeds. Reproduced on macOS with R 4.5, RcppParallel 6.2.0, and CmdStan 2.39.0 and 2.40.0-rc1, on cmdstanr master and on the v1.0 branch.

Why

Stan Math's autodiff core includes TBB headers, so the compiled methods reference symbols from the TBB CmdStan bundles, which has been 2020.3 in every release from 2.27 on. RcppParallel ships oneTBB, which dropped the tbb::internal namespace. Both are installed as libtbb.dylib, so once RcppParallel's copy is in the process the loader binds the new object to it and the symbol is missing. On Linux the two generations have different sonames (libtbb.so.2 and libtbb.so.12), so they may coexist there; not tested.

Sampling is unaffected: the executable runs in a child process. Only the model methods (log_prob(), grad_log_prob(), unconstrain_variables(), ...) and $expose_functions() are exposed, and only in a session that loaded rstan or brms first.

What to do

The real fix is upstream: CmdStan bundling oneTBB, the generation everything else now ships. Stan Math already builds against an external oneTBB, so this is a question of the bundled default.

In cmdstanr, check before building model methods whether a foreign TBB is already loaded (getLoadedDLLs() lists RcppParallel's libtbb) and error with a sentence saying so and suggesting a fresh session with the methods built before rstan or brms is loaded. That turns the linker dump above into an actionable message. Linking the methods against CmdStan's TBB by absolute path would avoid the collision but means rewriting install names per build, which is not worth it against the upstream fix.

Reproduce

loadNamespace("RcppParallel")
library(cmdstanr)
mod <- cmdstan_model(write_stan_file("
data { int<lower=0> N; array[N] int<lower=0,upper=1> y; }
parameters { real<lower=0,upper=1> theta; }
model { theta ~ beta(1,1); y ~ bernoulli(theta); }
"))
fit <- mod$sample(data = list(N = 10, y = c(0,1,0,0,0,0,0,0,0,1)), chains = 1)
fit$init_model_methods()

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the init_model_methods() and expose_functions() entry points, then run the reproduction after loadNamespace("RcppParallel") to observe the TBB loading failure. Add the requested pre-build check for RcppParallel's loaded libtbb and make the error explain the conflict and suggest a fresh session. Done means both entry points fail with that actionable message instead of the linker dump.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
tooling
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.