Adaptiveness of CMA based samplers

Abstract

We turn the Covariance Matrix Adaptation Evolution Strategy into an adaptive Markov Chain Monte Carlo (or MCMC) sampling algorithm that adapts online to the target distribution, i.e. the distribution to be sampled from. We call the resulting algorithm CMA-Sampling. It exhibits a higher convergence rate, a better mixing, and consequently a more effective MCMC sampler. We look at a few variants and compare their adaptiveness to a number of other adaptive samplers, including Haario et. al's AM sampler, on a testsuite of 4 target distributions.