stan-dev / stan-dev/math

integrate_1d_gauss_kronrod gradient computation can be very slow

Open
#3,392 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

  • #3393 by @avehtari — closed without merging
Dominant language
C++
Stars
839
Forks
220
Avg merge
2d 4h
Merged PRs (30d)
14

Description

Summary

The gradient computation for integrate_1d_gauss_kronrod call the Boost adaptive Gauss Kronrod again, but Boost allows only relative tolerance and that can lead the adaptation to spend much more time than what the quadrature for the actual integrand takes.

Problem

integrate_1d_gauss_kronrod_tol computes parameter gradients through
internal::integrate_1d_adjoint, which integrates ∂f/∂θ_i separately
for every var argument with the same integrator and tolerance as the
value (rev/functor/integrate_1d_adjoint.hpp, integrate_grad).
Boost's gauss_kronrod::integrate decides whether to bisect a leaf
against tol × |estimate of that leaf| (gauss_kronrod.hpp,
recursive_adaptive_integrate).

This is a purely relative rule. A gradient component whose integrand
is tiny relative to f and dominated by round-off can never satisfy
it: the K21 − G10 disagreement on noise is O(1) relative, so that
component bisects to max_depth (2^15 leaves × 21 nodes by default) on
every call, although its contribution to ∂ log I/∂θ_i = (∫∂f/∂θ_i)/I
is negligible. Stan's own post-check, error ≤ max(rel_tol × L1, abs_tol), is applied only after Boost returns, and Boost's public
integrate hard-codes the recursion's absolute budget to 0, so Stan's
absolute_tolerance cannot reach the bisection.

Where this arises in practice: a marginal likelihood
∫ N(z) ∏_k BetaBinomial(y_k | T_k, φ·logit⁻¹(η_k), φ·logit⁻¹(−η_k)) dz
with saturated observations (y_k = 0). In a tail panel where
f ~ 1e-12, the φ-score of a saturated observation is a difference of
large digamma terms that cancels catastrophically, so ∂f/∂φ ~ 1e-23
and noisy. Measured per subject with an evaluation counter (unpatched):
value 273 evaluations; gradients 668 850, of which 666 393 are that one
component in that one panel; the other seven components converge in 21.

integrate_1d (double-exponential) and integrate_1d_double_exponential are not affected: tanh_sinh /
sinh_sinh terminate on err ≤ tol × L1 at every level.

Measured on a beta-binomial varying-intercept model (105 per-subject
marginals, ~9 var arguments each; one gradient of the whole model at
the same unconstrained point, n = 10; CmdStan 2.39, -O3):

gradient vs integrate_1d max |Δ grad| vs integrate_1d
integrate_1d (DE) 56.6 ms 1
integrate_1d_gauss_kronrod_tol 7582 ms 134× 2.9e-11

In generated quantities (no gradients) the same GK integral costs 2.1×
DE, so the 134× is entirely the gradient path.

Fix

The quantity consumed downstream is ∂ log I/∂θ_i = (∫ ∂f/∂θ_i) / I,
so each gradient integral only needs absolute accuracy rel_tol × I.
Give Boost's relative rule that absolute scale by integrating

g_i(x) = ∂f/∂θ_i (x) + c f(x)

and returning ∫ g_i − c I, where I is the value integral already
computed and c is constant (not 0 or 1). Every leaf estimate of g_i is then ≈ the leaf's value
mass, so a noisy component that is small relative to f is accepted
as soon as the value's leaf is, and the bisection stops where the value
integral's did. The subtraction's cancellation error is rel_tol × I,
the accuracy the value has anyway. Guarded on I finite and non-zero
(otherwise unchanged behaviour), so a signed f with zero integral is
no worse than today.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in rev/functor/integrate_1d_adjoint.hpp at integrate_grad, then read Boost's gauss_kronrod.hpp and recursive_adaptive_integrate behavior. Compare the gradient path with integrate_1d using the saturated beta-binomial case described in the issue; done means avoiding excessive evaluations while preserving the reported gradient accuracy and existing behavior for non-finite or zero value integrals.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.