This paper presents a novel L1-norm semi-supervised learning algorithm for robust image analysis by giving new L1-norm formulation of Laplacian regularization which is the key step of graph-based semi-supervised learning. Since our L1-norm Laplacian regularization is defined directly over the eigenvectors of the normalized Laplacian matrix, we successfully formulate semi-supervised learning as an L1-norm linear reconstruction problem which can be effectively solved with sparse coding. By working with only a small subset of eigenvectors, we further develop a fast sparse coding algorithm for our L1-norm semi-supervised learning. Due to the sparsity induced by sparse coding, the proposed algorithm can deal with the noise in the data to some extent and thus has important applications to robust image analysis, such as noise-robust image classification and noise reduction for visual and textual bag-of-words (BOW) models. In particular, this paper is the first attempt to obtain robust image representation by sparse co-refinement of visual and textual BOW models. The experimental results have shown the promising performance of the proposed algorithm.
Semi-supervised learning, i.e., learning from both labeled and unlabeled data, has been widely applied to many challenging image analysis tasks [1]- [6] such as image representation, image classification, and image annotation. In different image analysis tasks, the manual labeling of training data is often tedious, subjective as well as expensive, while the access to unlabeled data is much easier. Through exploiting the large number of unlabeled data with reasonable assumptions, semisupervised learning [7]- [11] can reduce the need for expensive labeled data and thus achieve promising results especially for community-contributed image collections (e.g. Flickr).
Among various semi-supervised learning methods, one influential work is graph-based semi-supervised learning [8], [9] which models the entire dataset as a graph. The basic idea behind this semi-supervised learning is label propagation on the graph with the cluster consistency [9] (i.e. two data points on the same geometric structure are likely to have the same class label). Since the graph is at the heart of graph-based semi-supervised learning, graph construction has been extensively studied [12]- [15] in the past years. However, these graph construction methods are not developed directly for noise reduction, and the corresponding semi-supervised
The authors are with the Institute of Computer Science and Technology, Peking University, Beijing 100871, China (e-mail: luzhiwu@icst.pku.edu.cn, pengyuxin@icst.pku.edu.cn).
learning may suffer from significant performance degradation due to the inaccurate labeling of data points commonly encountered in different image analysis tasks. For example, the annotations of images may be contributed by the community (see Flickr) and we can only obtain noisy tags.
In this paper, we focus on proposing a novel noise-robust graph-based semi-supervised learning method, rather than the well-studied graph construction. As summarized in [12], the traditional graph-based semi-supervised learning can be formulated as a quadratic optimization problem based on Laplacian regularization [4], [8], [9], [11], [16]. Considering that the sparsity induced by L 1 -norm optimization can help to deal with the noise in the data to some extent [17], [18], if we succeed in formulating Laplacian regularization as an L 1 -norm term instead, we can convert the traditional semi-supervised learning to L 1 -norm optimization and enable our new semisupervised learning also to benefit from the nice property of sparsity. Fortunately, derived from the eigenvalue decomposition of the normalized Laplacian matrix L, we can readily represent L in a symmetrical decomposition form, which can be further used to formulate Laplacian regularization as an L 1 -norm term. Since all the eigenvectors of L are explored in this symmetrical decomposition, our new L 1 -norm Laplacian regularization can be considered to be explicitly formulated based upon the manifold structure of the data.
As a convex optimization problem, the above L 1 -norm semi-supervised learning has a unique global solution. By working only with a small subset of eigenvectors, we develop a fast sparse coding algorithm for our L 1 -norm semisupervised learning. In this paper, we only adopt the fast iterative shrinkage-thresholding method [19] for sparse coding, regardless of many other L 1 -norm optimization methods [20]- [23]. Due to the nice property of sparsity, the proposed algorithm can deal with the noise in the data to some extent, as shown in our later experiments. Hence, it has important applications to robust image analysis where noisy labels are provided. In this paper, we apply the proposed algorithm to two typical image analysis tasks, i.e., noise-robust semisupervised image classification and noise reduction for both visual and textual bag-of-words (BOW) models. Although only tested in these two applications, the proposed algorithm can be extended to other image analysis tasks, given that semisupervised learning has been widely used in the literature.
To emphasize the main contributions of this paper, we summarize the following distinct advantages of our novel L 1norm semi-supervised learning:
• We have made the first attempt to formulate Laplacian regularization as an L 1 -norm term explicitly based upon the manifold structure of the data. • Our L 1 -norm semi-supervised learning algorithm has been shown to achieve significant improvements in robust image analysis where noisy labels are provided. • Our new L 1 -norm Laplacian regularization can be similarly applied to many other difficult problems, considering the wide use of Laplacian regularization. • This is the first attempt to obtain robust image representation by sparse co-refinement of visual and textual BOW models for community-contributed image collections. The remainder of this paper is organized as follows. Section II provides a brief review of related work. In Section III, we propose a fa
This content is AI-processed based on open access ArXiv data.
