Clustering-based meta Bayesian optimization with theoretical guarantee
Abstract
Bayesian Optimization (BO) is a well-established method for addressing black-box optimization problems. In many real-world scenarios, optimization often involves multiple functions, emphasizing the importance of leveraging data and learned functions from prior tasks to enhance efficiency in the current task. To expedite convergence to the global optimum, recent studies have introduced meta-learning strategies, collectively referred to as meta-BO, to incorporate knowledge from historical tasks. However, in practical settings, the underlying functions are often heterogeneous, which can adversely affect optimization performance for the current task. Additionally, when the number of historical tasks is large, meta-BO methods face significant scalability challenges. In this work, we propose a scalable and robust meta-BO method designed to address key challenges in heterogeneous and large-scale meta-tasks. Our approach (1) effectively partitions transferred meta-functions into highly homogeneous clusters, (2) learns the geometry-based surrogate prototype that capture the structural patterns within each cluster, and (3) adaptively synthesizes meta-priors during the online phase using statistical distance-based weighting policies. Experimental results on real-world hyperparameter optimization (HPO) tasks, combined with theoretical guarantees, demonstrate the robustness and effectiveness of our method in overcoming these challenges.
RAS ID
83666
Document Type
Conference Proceeding
Date of Publication
1-1-2025
Volume
15872 LNAI
School
School of Science
Copyright
subscription content
Publisher
Springer
Identifier
Viet Huynh: https://orcid.org/0000-0003-1308-1164
Comments
Nguyen, K., Huynh, V., Tran, B., Pham, T., Huynh, T., & Nguyen, T. (2025). Clustering-based meta Bayesian optimization with theoretical guarantee. Lecture Notes in Computer Science, 210–223. https://doi.org/10.1007/978-981-96-8180-8_17