Effectiveness of Random Search in SVM hyper-parameter tuning

Abstract

Classification is one of the most common machine learning tasks. SVMs have been frequently applied to this task. In general, the values chosen for the hyper-parameters of SVMs affect the performance of their induced predictive models. Several studies use optimization techniques to find a set of hyper-parameter values that induces classifiers with good predictive performance. This paper investigates the hypothesis that a simple Random Search method is sufficient to adjust the hyper-parameters of SVMs. A set of experiments compared the performance of five tuning techniques: three meta-heuristics commonly used, Random Search and Grid Search. The experimental results show that the predictive performance of models using Random Search is equivalent to those obtained using metaheuristics and Grid Search, but with a lower computational cost.