Transductive Learning for Retrieving and Mining Visual Contents

supported by NSF IIS-0308222

Contemporary visual learning methods for visual content mining tasks are plagued by several critical and fundamental challenges: (1) the unavailability of large annotated datasets prevents effective supervised learning; (2) the variability in different working environments challenges the generalization of inductive learning approaches; and (3) the high-dimensionality of these tasks confronts the efficiency of many existing learning techniques. The goal of this research project is to overcome these challenges by exploring a novel transductive learning approach.

The approach provides a unified framework accommodating four subtasks:

Transduction that integrates unlabelled and labelled data to alleviate the challenge of limited supervision and to enable automatic annotation propagation;
Model Transduction that automatically adapts a learned model to untrained environments for efficient model reuse;
Co-transduction that facilitates transduction with multi-modalities to handle high-dimensionality in visual data;
Co-inference that exploits the interactions among multiple modalities to enable efficient model transduction.

We have studied some theoretical issues based on Markov networks and fixed-point theory, while some are still open. We have also obtained some successful case studies, including content-based image retrieval, object categorization, model adaptation, cue integration, and mining co-location visual patterns.

The results of this project will lead to significant improvement on the quality of content-based and object-level multimedia retrieval, will greatly benefit visual recognition that requires large datasets for training and evaluation, will significantly reduce the efforts of training brand new models for un-trained scenarios, and will be very useful in intelligent video surveillance applications thus having a great impact on homeland security.

Research Team:

Ying Wu PI
Gang Hua (Ph.D. 2006, now a Scientist at Microsoft Live labs Research)
Ting Yu (Ph.D. 2006, now a Member of Technical Staff at GE Global Research)
Zhimin Fan (M.S. 2006, now an analyst at Beijing Development Bank)
Ming Yang
Junsong Yuan

Some Research Demos: (click to enter)

[1]	Model Transduction
[2]	Belief Inconsistency in Information Integration
[3]	Video data mining
[4]	Object detetction in images/video

Data and Sharing Information:

We have the following video databases that are available to the research community:

A video database for model transduction. The database has 10 long sequences, each of which exhibits the visual appearance changes of the main target of the video (e.g., a head, a human body, a watch, etc). The size of this database is about 2GB.
A video database for on-line video data mining. The database has about 100 long sequences of amateur video by using hand-held cameras. Each sequence has a major theme target, e.g., a kid. The purpose of this database is to evaluate the method of on-line video data mining in tracking the main target. The size of this database is about 20GB.
An image database for human detection. The database has about 2000 images of pedestrians, and another 3000 images of non-pedestrians. It was used to train a pedestrian detector. This size of this database is 2G.

These databases are available to the research community. Due to its volume, the current file server of our web is unable to provide enough space for the data. Alternatively, we can send DVDs of the data to those who are interested in them. In the future, we plan to add more disk space to the file server so that the data can be directly downloaded.

Publication:

Book Chapters

Qi Tian, Ying Wu, Jerry Yu and Thomas S. Huang, "Self-Supervised Learning Based on Discriminative Nonlinear Features and Its application for Image Retrieval", in Managing Multimedia Semantics, edited by Uma Srinivasan and Surya Nepal, ICT Centre CSIRO, Australia, by Idea Group, Inc., 2004.
Junsong Yuan and Ying Wu, "Common Pattern Discovery in Multimedia Data Mining", in Encyclopedia of Data Warehousing and Mining (2nd Edition), Edited by J. Wang, Idea Group Inc., 2008

Jornal Papers

Ming Yang, Gang Hua and Ying Wu, "Context-Aware Visual Tracking", IEEE Trans. on Pattern Analysis and Machine Intelligence, 2008
Zhimin Fan, Ming Yang and Ying Wu, "Multiple Collaborative Kernel Tracking", IEEE Trans. on Pattern Analysis and Machine Intelligence, vol.29, No.7, pp.1268-1273, July 2007
Ying Wu and Ting Yu, "A Field Model for Human Detection and Tracking", IEEE Trans. on Pattern Analysi s and Machine Intelligence, Vol.28, No.5., pp.753-765, May, 2006. [PDF]
Zhimin Fan, Jie Zhou and Ying Wu, "Inference of Multiple Subspaces from High Dimensional Data Using Oriented-Frames with Application to Multibody Grouping", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.28, No.1, pp.90-105, Jan., 2006. [PDF]
Gang Hua and Ying Wu, "Variational Maximum a Posteriori by Annealed Mean Field Analysis", IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.27, No.11, pp.1747-1781, Nov., 2005. [PDF]
Gang Hua and Ying Wu, "Sequential Mean Field Variational Analysis of Structured Deformable Shapes", Computer Vision and Image Understanding, Vol.101, No.2., pp.87-99, Feb., 2006. [PDF]
Ying Wu and Thomas S. Huang, "Robust Visual Tracking by Integrating Multiple Cues Based on Co-Inference Learning", Int'l Journal Computer Vision, vol.58, No.1, June, 2004. [PDF]
Qi Tian, Ying Wu, Jerry Yu and Thomas S. Huang, "Self-Supervised Learning Based on Discriminative Nonlinear Features and Its Applications for Pattern Classification", Pattern Recognition, Vol.38, No.6, 2005. [PDF]
Ying Wu and Thomas S. Huang, "Towards Self-Exploring Discriminating Features for Visual Learning", Journal of Engineering Application on Artificial Intelligence, Vol.15, pp.139-150,2002. [PDF]

Conference Papers

Junsong Yuan, Jiebo Luo and Ying Wu, "Mining Compositional Features for Boosting", in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR'08), Anchorage, Alaska, June 2008.
Junsong Yuan and Ying Wu, "Context-aware Clustering", in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR'08), Anchorage, Alaska, June 2008.
Jingjing Meng, Junsong Yuan, Mat Hans and Ying Wu, "Mining Motifs from Human Motion", in EuroGraphics, 2008, Crete, Greece, April 2008.

Ming Yang, Junsong Yuan and Ying Wu, "Spatial Selection for Attentional Visual Tracking", in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR'07), Minneapolis, MN, June 2007

Junsong Yuan, Ying Wu and Ming Yang, "Discovery of Collocation patterns: from Visual Words to Visual Phrases", in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR'07), Minneapolis, MN, June 2007

Junsong Yuan, Ming Yang and Ying Wu, "From Frequent Itemsets To Semantically Meaningful Visual Patterns", in Proc. ACM Int'l Conf. on Knowledge Discovery and Data Mining (SIGKDD'07), San Jose, CA, August 2007

Ming Yang, Ying Wu and Shihong Lao, "Mining Auxiliary Objects for Tracking by Multibody Grouping", in Proc. IEEE Int'l Conf. on Image Processing (ICIP'07), San Antonio, TX, Sept. 2007

Junsong Yuan, Zhu Li, Yun Fu, Ying Wu and Thomas S. Huang, "Common Spatial Pattern Discovery by Efficient Candidate Pruning", in Proc. IEEE Int'l Conf. on Image Processing (ICIP'07), San Antonio, TX, Sept. 2007

Ming Yang, Ying Wu and Shihong Lao, "Intelligent Collaborative Tracking by Mining Auxiliary Objects", in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR 06), New York City, NY, June 17-22, 2006.   [PDF]

Zhimin Fan, Ming Yang, Ying Wu, Gang Hua and Ting Yu, "Efficient Optimal Kernel Placement for Reliable Visual Tracking", in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR 06), New York City, NY, June 17-22, 2006.   [PDF]

Gang Hua, Ying Wu and Zhimin Fan, "Measurement Integration Under Inconsistency for Robust Tracking", in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR 06), New York City, NY, June 17-22, 2006.   [PDF]

Zhimin Fan and Ying Wu, "Multiple Collaborative Kernel Tracking", in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR'05), San Diego, CA, June 20-26, 2005.   [PDF]

Ying Wu, Ting Yu and Gang Hua, "A Statistical Field Model for Pedestrian Detection", in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR'05), San Diego, CA, June 20-26, 2005.   [PDF]

Ming Yang and Ying Wu, "Tracking non-stationary appearances and dynamic feature selection", in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR'05), San Diego, CA, June 20-26, 2005. [PDF]

Ting Yu and Ying Wu, "Decentralized Multiple Target Tracking using Netted Collaborative Autonomous Trackers", in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR'05), San Diego, CA, June 20-26, 2005. [PDF]

Ting Yu and Ying Wu, "Collaborative Tracking of Multiple Targets", in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR'04), vol.I, pp. 834-841, Washington, DC, June, 2004. [PDF]

Gang Hua and Ying Wu, "Multi-scale Visual Tracking by Sequential Belief Propagation", in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR'04), vol.I, pp. 826-833, Washington, DC, June, 2004. [PDF]

Gang Hua, Ying Wu and Ting Yu, "Analyzing Structured Deformable Shapes Via Mean Field Monte Carlo", in Proc. IEEE Asia Conference on Computer Vision ( ACCV'2004), Jeju Island, Korea, Jan., 2004.   [PDF]

Qi Tian and Jerry Yu and Ying Wu and Thomas S. Huang, "Learning Based on Kernel Discriminant-EM Algorithm for Image Classification", in Proc. IEEE International Conferenece On Acoustics, Speech, and Signal Processing (ICASSP'04), Montreal, Canada, 2004.   [PDF]

Ying Wu, Gang Hua and Ting Yu, "Tracking Articulated Body by Dynamic Markov Network", in Proc. IEEE Int'l Conf. on Computer Vision (ICCV'03), pp.1094-1101, Nice, France, Oct., 2003. [PDF]