0.2.0

Highlights

• Interface Changes to MCMC and SVI: The interface for inference algorithms have been simplified, and is much closer to Pyro. See MCMC and SVI.
• Multi-chain Sampling for MCMC: There are three options provided: parallel (default), sequential, and vectorized. Currently, parallel method is the fastest among the three.

Breaking changes

• The primitives param, sample are moved to primitives module. All primities are exposed in numpyro namespace.

New Features

MCMC

• In MCMC, we have the option to collect fields other than just the samples such as number of steps or step size, using collect_fields arg in MCMC.run. This can be useful when gathering diagnostic information during debugging.
• diverging field is added to HMCState. This field is useful to detect divergent transitions.
• Support improper prior through param primitives. e.g.

def model(data):
    loc = numpyro.param('loc', 0.)
    scale = numpyro.param('scale', 0.5, constraint=constraints.positive)
    return numpyro.sample('obs', dist.Normal(loc, scale), obs=data)

Primitives / Effect Handlers

• module primitive to support JAX style neural network. See VAE example.
• condition handler for conditioning sample sites to observed data.
• scale handler for rescaling the log probability score.

Optimizers

JAX optimizers are wrapped in the numpyro.optim module, so that the optimizers can be passed in directly to SVI.

Distributions

• New distributions: Delta, GaussianRandomWalk, InverseGamma, LKJCholesky (with both cvine and onion methods for sampling), MultivariateNormal.
• New transforms: CorrCholeskyTransform (which is vectorized), InverseAutoregressiveTransform, LowerCholeskyTransform, PermuteTransform, PowerTransform.

Utilities

• predictive utility for vectorized predictions from the posterior predictive distribution.

Autoguides

An experimental autoguide module, with more autoguides to come.

New Examples

• Sparse Linear Regression - fast Bayesian discovery of pairwise interactions in high dimensional data.
• Gaussian Process - sample from the posterior over the hyperparameters of a gaussian process.
• HMC on Neal's Funnel - automatic reparameterization through transform distributions.

Enhancements and Bug Fixes

• Improve compiling time in MCMC.
• Better PRNG splitting mechanism in SVI (to avoid reusing PRNG keys).
• Correctly handle models with dynamically changing distribution constraints. e.g.

def model():
    x = numpyro.sample('x', dist.Uniform(0., 2.))
    y = numpyro.sample('y', dist.Uniform(0., x))  # y's support is not static.

• Fixes step_size getting NaN in MCMC when it becomes extremely small.