Model Adaptation and Dynamic Feature Selection

Model adaptation is a fundamental issue in tracking. Since the changes of visual appearances jeopardize the measurement model and fail the tracker, trackers need to be adaptive to non-stationary appearances. However, the risk is the adaptation drift. We obtained a closed-form solution to subspace adaptation based on novel constraints such as negative data, pair-wise constraints, and adaptation dynamics.

Visual tracking could be treated as a parameter estimation problem of target representation based on observations in image sequences. A richer target representation would incur better chances of successful tracking in cluttered and dynamic environments. However, the dimensionality of target's state space also increases making tracking a formidable estimation problem. The problem of tracking and integrating multiple cues is formulated in a probabilistic framework and represented by a factorized graphical model. Structured variational analysis of such graphical model factorizes different modalities and suggests a co-inference process among these modalities. Based on this theoretical analysis, we design a sequential Monte Carlo algorithm that runs real-time at around 30Hz.

Demo Sequences


Subspace adaptation (partial occlusion)	Subspace adaptation (360 out-of-plane rotation)	Subspace adaptation (walking)	Subspace adaptation (in-plane ratation)

Co-inference (girl)	Co-inference (hand)	Co-inference (speaker)	Co-inference (VR)

In these sequences, the target (i.e., the head) exhibits very large variations in its visual appearance in each sequence, as the head rotates. It is very difficult to use a fixed and complex visual appearance model to track the target in all views, because of the large undertainty in the appearance changes. An alternative is to adapt a simpler model for different views. Although this seems to be plausible, adaptation drift is quite common, and thus a very careful treatment is needed (for details please see our papers).

Publication:

Ying Wu and Thomas S. Huang, "Robust Visual Tracking by Integrating Multiple Cues Based on Co-Inference Learning", Int'l Journal Computer Vision (IJCV), Vol.58, No. 1, June 2004. [PDF]

Ming Yang and Ying Wu, "Tracking non-stationary appearances and dynamic feature selection", in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR'05), San Diego, CA, June 20-26, 2005. [PDF]

Ying Wu and Thomas S. Huang, "A Co-inference Approach to Robust Visual Tracking", in Proc. IEEE Int'l Conf. on Computer Vision (ICCV'01), Vol.II, pp.26-33, Vancouver, Canada, July, 2001. [PDF]

Ying Wu, Thomas S. Huang, "Color Tracking by Transductive Learning", in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR'2000), Vol.I, pp. 133-138, Hilton Head Island, SC, June, 2000. [PDF]

Return to Tracking Research