PROJECT TITLE :

Robust Variational Learning for Multiclass Kernel Models With Stein Refinement

ABSTRACT:

The ability of kernel-based models to generalize well is impressive, but the vast majority of them, including the SVM, are susceptible to the "curse of kernelization." In addition, the accuracy of their predictions is highly dependent on the hyperparameter tuning, which requires a significant amount of computational resources. When it comes to dealing with large-scale datasets, these problems make using kernel methods problematic. To achieve this goal, we first formulate the optimization problem in a kernel-based learning setting as a posterior inference problem. Next, we develop a diverse family of variational inference techniques that are based on Recurrent Neural Networks. In contrast to the previous research, which stops at the variational distribution and uses it as a surrogate for the distribution of the true posterior, the current study makes use of Stein Variational Gradient Descent in order to bring the variational distribution even closer to the distribution of the true posterior. We refer to this step as Stein Refinement. After considering all of these factors, we have developed a variational learning method that is reliable, effective, and capable of producing extremely accurate approximations for multiclass kernel machines. In addition, our formulation makes it possible to learn kernel parameters and hyperparameters in an effective manner, which strengthens the proposed method against the effects of uncertain data. The exhaustive experiments demonstrate that our method achieves accuracy that is comparable to that of LIBSVM, a well-known implementation of SVM, and that outperforms other baselines, all while having the ability to scale with large-scale datasets in a seamless manner. This was achieved without tuning any parameters on modest quantities of data.