Bayesian Optimization: Surrogate Models, Acquisition Functions, and Practical Implementation
Problem Setting Consider the task of minimizing an expensive, black-box function \(f: \mathcal{X}\to\mathbb{R}\) where the analytic form is unknown and only point-wise evaluations are available. In machine-learning terms, this is the hyper-parameter tuning problem: given a validation metric \(R^2(\t...