Self-supervised Learning for Content-based Image Retrieval

Content-based image retrieval is to retrieve similar images to the query images based on image contents. It involves some fundamental problems of image understanding. Previous approaches focused on investigating the similarity measurements or metric of different images. Many heuristic methods have been proposed. We treat this problem as a learning problem by learning the similarities from examples. A classifier could be trained to classify relevant and irrelevant images. However, one of the difficulties is that only a very small number of query images are available as supervised data. It is infeasible to learn meaningful concepts from such a small data set.

Our idea is to integrating the image database in learning, i.e., learning is based on both supervised query images and unsupervised database images. The intuition is that the database with a large number of unlabeled images could characterize a local probability density that could be helpful for the hybrid learning. Obviously, if an image is not in the database, why do need to classify this image? On the other hand, our learning is only interested in the images in the database, i.e., a local distribution in the data space. Our proposed Discriminant-EM (D-EM) algorithm is very suitable for this problem. D-EM inserts a discrimination step in the EM iteration, to find a discriminating subspace to simplify the probabilistic structure of the data distribution. D-EM has achieved very promising results.
 

Return to Video Research Updated 5/2006. Copyright © 2001-2006 Ying Wu