Bayesovska optimizacija hiperparametrov v strojnem učenju ; Bayesian Optimization of Hyperparameters in Machine Learning

Cilj naše diplomske naloge je bil analizirati Bayesovsko optimizacijo na problemu optimizacije hiperparametrov. Podlaga za analizo sta pogosto uporabljani orodji za optimizacijo hiperparametrov: naključno iskanje in iskanje v mreži. Predstavimo Bayesovsko optimizacijo, s poudarkom na Gaussovih procesih in odločilnih funkcijah: EI, PI in LCB. Izvedemo deset eksperimentov, pri katerih optimiziramo hiperparametre petih različnih modelov. Pri eksperimentih analiziramo dve zelo pomembni metriki: hitrost optimizacije in rezultate, ki jih doseže optimizirani model. Modeli optimizirani z Bayesovsko optimizacijo so v povprečju v primerljivem času dosegli boljše rezultate kot tisti, ki so bili optimizirani z naključnim iskanjem in iskanjem v mreži. ; The goal of our thesis was to analyze Bayesian optimization on the problem of hyperparameter optimization. The basis for the analysis is the commonly used tools for hyperparameter optimization: random search and grid search. We introduce Bayesian optimization with an emphasis on Gaussian processes and acquisition functions: EI, PI, LCB. We perform ten experiments in which we optimize the hyperparameters of five different models. In experiments, we analyze two really important metrics: speed of optimization and results that the optimized model achieved. Models optimized with Bayesian optimization in comparable time achieved better results on average than those that were optimized with random search and grid search.