Release of CmdStan 2.26.0

We are very happy to announce that the 2.26.0 release of CmdStan is now available on Github! As usual, the release of CmdStan 2.26.0 is accompanied with the new releases of Stan Math, core Stan and Stanc3.

This new release brings bug fixes, improvements for robustness, a few new functions, performance improvements and profiling.

Contributors

We would like to thank everyone that contributed to this release.

In total, we had code contributions from 19 developers and 10 additional users that contributed to our user manuals, guides and other documentation. Listing their Github usernames in alphabetical order: abeeisnotabug, adamhaber, Alanocallaghan, andrjohns, bbbales2, Dashadower, dirmeier, hsbadr, hyunjimoon, jgabry, jsocolar, mandel, mcol, mike-lawrence, mitzimorris, MKyhos, nhuurre, nicokist, pgree, rok-cesnovar, rybern, sakrejda, , seantalts, serban-nicusor-toptal, spinkney, SteveBronder, t4c1, wds15 and yizhang-yiz.

New Stan developers

We would also like to welcome four new Stan developers that accepted the invitation to formally join our Stan development team: Hyunji Moon (hyunjimoon), Sean Pinkney (spinkney), Shinyoung Kim (Dashadower) and Simon Dirmeier (dirmeier).

A big thank you to all four for your contributions so far and we are excited to see what comes next.

Bug fixes and miscellaneous improvements

• Improved robustness for boundary values of negative binomial and 2F1 functions.
• Fixed a bug with integrate_1d tolerances.
• Improved ordered_probit_lpmf for increased numerical stability.
• Support building native CmdStan on M1 macOS.
• Updated mdivide_left to avoid doing QR decomposition in reverse passes.
• Upgraded downstream libraries: now using Eigen 3.3.9 and Sundials to 5.6.1.
• Switched to using immediately invoked lambdas in size and range error checks to improve code caching.
• Added symmetric positive definite checks for inputs of Wishart and inverse Wishart functions.
• Fixed potential race condition when using print in threaded reduce_sum.
• Fixed a bug in dirichlet_lpdf and multi_normal_cholesky_lpdf, where function overloads that do broadcasting produced wrong derivatives.
• Replaced the use of Boost Program options with the header-only CLI11 library to make the building process more robust.

New functions

This release brings six new functions and a new distributions to the Stan language. Read their documentation by clicking the links:

• chol2inv
• discrete_range distribution
• generalized_inverse
• linspaced_int_array
• svd_U, svd_V
• symmetrize_from_lower_tri

Profiling

Users can now profile the computational costs (execution time and memory consumption) of their models using profiling statements as shown in this example:

model {
   matrix[N, N] cov;
   matrix[N, N] L_cov;
   profile("building_cov") {
      cov =   gp_exp_quad_cov(x, alpha, rho)
                        + diag_matrix(rep_vector(sigma, N));
   }
   profile("cholesky_cov"){
      L_cov = cholesky_decompose(cov);
   }   
   profile("priors"){
      rho ~ gamma(25, 4);
      alpha ~ normal(0, 2);
      sigma ~ normal(0, 1);
   }
   profile("likelihood") {
      y ~ multi_normal_cholesky(rep_vector(0, N), L_cov);
   }
}

Stan will report the execution time and memory consumption of automatic differentiation (AD) for the statements in the profile. The two main motivations for using profiling is to identify bottlenecks in a model and to evaluate different approaches for a section of a model. The former can help preventing premature optimization of sections with negligible computational costs.

Without profiling, users had to have a good and deep understanding of the underlying AD implementation in Math for all the functions in order to try and optimize a section of the model using different approaches (using other functions or reduce_sum for example). But even then it was very hard to measure the effect of a change to a section of the model as the effect was hard to identify in the execution time reported for a model’s overall execution which were very noisy due to randomness of sampling as well as input/output.

See here for a profiling example using CmdStan, and here for a profiling example using CmdStanR (requires the latest Github version of CmdStanR).

Backend performance improvements

Improved use of Eigen expressions

All primitive functions in Stan Math (/prim folder) as well as user-defined Stan functions now accept and use Eigen expressions. Where applicable they also return Eigen expressions. This change has been in the works for multiple release cycles now and is now finalized.

Users should experience increased performance of generated quantites for statements with multiple vector/row_vector/matrix operations and increased performance of some distributions in all blocks of a Stan model.

Increased and more robust performance of reduce_sum

We have optimized memory handling and copies when using automatic differentiation with reduce_sum.

Improved error messaging in the compiler and deprecation warnings

All functions and language features that have been deprecated now raise a warning during
compilation. You can use deprecated features/functions as normal until the next major release of Stan, but we advise you move to using the suggested non-deprecated function/language feature as soon as possible.

The Stan-to-C++ transpiler also now has an improved syntax/semantic error reporting.

OpenCL support for additional distributions

In addition to the GLM lpdf/lpmf functions, there are now 32 additional distributions that can utilize OpenCL on the GPU or CPU to speedup execution. The speedups vary between distributions and argument types (whether they are data or parameters). We have observed speedups in the range of 2- to 50-fold compared to sequential execution. We anticipate the next Stan release in April to have a fully-fledged GPU-enabled backend support for most Stan functions and any feedback at this point would be more than welcome.

We have also upgraded our OpenCL support for CPUs and integrated GPUs, limiting transfers when those devices are used.

See here for instructions on how to setup and use OpenCL with CmdStan.

How to install the new release?

Download the tar.gz file from the CmdStan release page, extract it and use it the way you use any Cmdstan release. We also have an online Cmdstan guide available here for new users.

If you are using cmdstanpy install the new release using

import cmdstanpy
cmdstanpy.install_cmdstan()

With CmdStanR you can install the new release using

library(cmdstanr)
install_cmdstan(cores = 4)