An Adapted GRASP Approach for Hyperparameter Search on Deep Networks Applied to Tabular Data
Photo by AuthorsThe robustness and resilience of the deep learning models offer consistent and competitive results in real-world applications. Despite its adaptability, the training and adjustment of the hyperparameters still demand knowledge and time from the designer. This paper proposes a simple and effective approach based on the Greedy Randomized Adaptive Search Procedure (GRASP) algorithm that we adapt to optimize deep neural networks models. We evaluated the performance of the proposed approach using the models Deep Feedforward Neural Network (DFNN) and TabNet, considering the Tabu Search algorithm as a baseline in five tabular datasets. Both optimization algorithms showed high performance regarding the (i) quality of the best solution, (ii) convergence, and (iii) local search. However, the adapted GRASP approach showed better results, optimizing the deep models in all datasets with statistical significance.
