Speaker
Description
In this work, a hyperparameter (kernel and C) optimization model for a support vector machine (SVM) classifier applied to handwritten digit recognition is presented. The kernel determines how the data is transformed and separated, which directly affects the model’s ability to capture complex patterns. The regularization parameter controls the balance between fitting the training data and maintaining the model’s ability to generalize to new data.
We developed a bilevel optimization framework where the lower level minimizes the SVM loss using a deterministic algorithm (L-BFGS-B), and the upper level searches for the optimal hyperparameter values using a particle swarm optimization metaheuristic. Cross-validation with three folds is used to evaluate model performance, reporting mean accuracy and standard deviation.
We compare the bilevel approach with other automated hyperparameter tuning methods, including grid search, random search, Hyperband, and Bayesian optimization. Preliminary results suggest that the bilevel framework can achieve superior classification performance, although the tradeoff between the quality of results and the computational effort should be further investigated. These experiments highlight the potential of bilevel optimization for tuning hyperparameters in complex machine learning models.
We acknowledge the support of the HarmonicAI project, which contributed to the development of this work.
