Two paper are accepted by Neurocomputing Journal.

Hessian Regularization by Patch Alignment Framework

Abstract

In recent years, semi-supervised learning has played a key part in large-scale image management, where usually only a few images are labeled. To address this problem, many representative works have been reported, including transductive SVM, universum SVM, co-training and graph-based methods. The prominent method is the patch alignment framework, which unifies the traditional spectral analysis methods. In this paper, we propose Hessian regression based on the patch alignment framework. In particular, we construct a Hessian using the patch alignment framework and apply it to regression problems. To the best of our knowledge, there is no report on Hessian construction from the patch alignment viewpoint. Compared with the traditional Laplacian regularization, Hessian can better match the data and then leverage the performance. To validate the effectiveness of the proposed method, we conduct human face recognition experiments on a celebrity face dataset. The experimental results demonstrate the superiority of the proposed solution in human face classification.

Keywords—semi-supervised learning; Hessian; patch alignment; Least Squares

HSAE: A Hessian Regularized Sparse Auto-Encoders

Abstract

Auto-encoders are one kinds of promising non-probabilistic representation learning paradigms that can efficiently learn stable deterministic features. Recently, auto-encoder algorithms are drawing more and more attentions because of its attractive performance in learning insensitive representation with respect to data changes. The most representative auto-encoder algorithms are the regularized auto-encoders including contractive auto-encoder, denoising auto-encoders, and sparse auto-encoders. In this paper, we incorporate both Hessian regularization and sparsity constraints into auto-encoders and then propose a new auto-encoder algorithm called Hessian regularized sparse auto-encoders (HSAE). The advantages of the proposed HSAE lie in two folds: (1) it employs Hessian regularization to well preserve local geometry for data points; (2) it also efficiently extracts the hidden structure in the data by using sparsity constraints. Finally, we stack the single-layer auto-encoders and form a deep architecture of HSAE. To evaluate the effectiveness, we construct extensive experiments on the popular datasets including MNIST and CIFAR-10 dataset and compare the proposed HSAE with the basic auto-encoders, sparse auto-encoders, Laplacian auto-encoders and Hessian auto-encoders. The experimental results demonstrate that HSAE outperforms the related baseline algorithms.

Keywords: Hessian regularization; Sparse Representation; Auto-Encoder; Manifold