Skip to content

Review: Taking the Human Out of the Loop: A Review of Bayesian Optimization

Citation

  • Shahriari, B., Swersky, K., Wang, Z., Adams, R. P. & de Freitas, N. (2016). Taking the human out of the loop: a review of Bayesian optimization. Proceedings of the IEEE, 104(1), 148–175.
  • DOI

Abstract

Bayesian optimization is a framework for the global optimization of expensive, black-box objective functions that are noisy and lack accessible derivatives. It builds a probabilistic surrogate model of the objective from the evaluations seen so far, and uses an acquisition function derived from that model to choose the next point to evaluate, trading exploration of uncertain regions against exploitation of promising ones. This review introduces the framework, surveys statistical models and acquisition functions, and showcases applications from hyperparameter tuning to experimental design.

Bayesian optimization targets the problem \( \mathbf{x}^\star = \arg\max_x f(\mathbf{x}) \) when \( f \) is expensive to evaluate, has no closed form or derivatives, and returns noisy values. Because each evaluation is costly, the aim is to spend as few as possible, using the whole history to decide where to look next.

The framework has two ingredients. The first is a probabilistic surrogate model, a prior over objective functions (typically a Gaussian process) updated by Bayes' rule into a posterior as evaluations accumulate; the posterior gives a predicted mean and an uncertainty at every point. The second is an acquisition function \( \alpha_n(\mathbf{x}) \) computed from that posterior, which scores candidate points and is itself cheap to optimize. The next evaluation is placed at \( \mathbf{x}_{n+1} = \arg\max_x \alpha_n(\mathbf{x}) \), the objective is queried, the data are augmented, and the model is updated. Acquisition functions such as probability of improvement, expected improvement, upper confidence bound, Thompson sampling, and entropy search differ in how they weigh exploitation (high predicted mean) against exploration (high predicted uncertainty).

Relevance to Hifuku

Bayesian optimization is the model-based end of the exploration spectrum, and it shares Hifuku's setting: the objective is expensive to evaluate. Hifuku's performance is the alignment log-likelihood, and each evaluation is a full Felsenstein pass, so spending evaluations wisely matters.

The methods diverge in how they do so, which is the useful contrast. Bayesian optimization fits a surrogate to the objective and lets an acquisition function's uncertainty term drive exploration toward a single optimum. Hifuku builds no surrogate: it measures the log-likelihood directly at every tree it visits, and its diversity comes from the niche structure of the elite archive rather than from a model's predicted uncertainty. It is also a survey, not an optimizer, so it keeps the best tree in every niche instead of converging on one maximum. A surrogate is most valuable when the objective cannot be evaluated at a point, and Hifuku is in the opposite situation, where the elevation is a direct computation. Bayesian optimization therefore names the alternative Hifuku does not take, and it belongs here as context for the exploration stance rather than as a component of the engine.