Intelligent Systems Lab

Reseach related to human body
Data-free Prior Model for Upper Body Pose Estimation and Tracking
Video based human body pose estimation seeks to estimate the human body pose from an image or a video sequence, which captures a person exhibiting some activities. In order to handle noise and occlusion, a pose prior model is often constructed and the prior model is subsequently combined with the pose estimated from the image data to achieve a more robust body pose tracking. Various body prior models have been proposed. Most of them are data-driven, typically learned from the 3D motion capture data. In addition to being expensive and time-consuming to collect, these data-based prior models cannot generalize well to activities and subjects not present in the motion capture data. To alleviate this problem, we propose to learn the prior model from basic principles that govern the body pose and its movement, rather than from the motion capture data.
  • These principles are derived from anatomy, biomechanics and physics theories. For this, we propose a framework and methods that can simultaneously capture different types of domain knowledge and systematically embed them into the prior model.
  • Experiments on benchmark datasets show the proposed prior model, compared with current data-based prior models, achieves comparable performance for body motions that are present in the training data. It, however, significantly outperforms the data-based prior models in generalization to different body motions and to different subjects.
  • The proposed framework can be applied to other vision problems, where the required training data is limited or difficult to acquire but much domain knowledge is readily available.

Publication: To be appeared in IEEE Transaction on Image Processing, 2013

Switching Gaussian Process for Motion Tracking
In this work, we proposed the marriage of the switching dynamical system and recent Gaussian Process Dynamic Models (GPDM), yielding a new model called the switching GPDM (SGPDM).
  • The proposed switching variables enable the SGPDM to capture diverse motion dynamics effectively, and also allow to identify the motion class(e.g. walk or run in the human motion tracking, smile or angry in the facial motion tracking), which naturally leads to the idea of simultaneous motion tracking and classi?cation.
  • Each of GPDMs in SGPDM can faithfully model its corresponding primitive motion, while performing tracking in the low-dimensional latent space, therefore signi?cantly improving the tracking ef?ciency.
  • The proposed SGPDM is then applied to human body motion tracking and classi?cation, and facial motion tracking and recognition. We demonstrate the performance of our model on several composite body motion videos obtained from the CMU database, including exercises and salsa dance.
  • We also demonstrate the robustness of our model in terms of both facial feature tracking and facial expression/pose recognition performance on real videos under diverse scenarios including pose change, low frame rate and low quality videos.

Publication: Jixu Chen, Minyoung Kim, Yu Wang, and Qiang Ji, Switching Gaussian Process Dynamic Models for Simultaneous Composite Motion Tracking and Recognition, IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2009.PDF

3D Upper Body Tracking with Self-Occlusions
We proposed an efficient 3D upper body tracking method, which recovers the positions and orientations of six upper-body parts from the video sequence. Our method is based on a probabilistic graphical model(PGM), which incorporates the spatial relationships among the body parts, and a robust multi-view image likelihood using probabilistic PCA (PPCA). For the ef?ciency, we use a tree-structured graphical model and use the particle based belief propagation to perform the inference. Since our image likelihood is based on multiple views, we address the self-occlusion by modeling the likelihood of the body part in each view, and automatically decrease the in?uence of the occluded view in the inference procedure.

Publication: Jixu Chen and Qiang Ji, Efficient 3D Upper Body Tracking with Self-Occlusions, International Conference on Pattern Recognition, 2010.PDF

Dynamic Bayesian Network (DBN) for Upper Body Tracking
We proposed a Dynamic Bayesian Network (DBN) model for upper body tracking.
  • We first construct a Bayesian Network (BN) to represent the human upper body structure and then incorporate into the BN various generic physical and anatomical constraints on the parts of the upper body. Unlike the existing upper body models, ours aims at handling physically feasible body motion rather than only some typical motion patterns.
  • We also explicitly model part self-occlusion in the DBN model, which allows to automatically detect the occurrence of self-occlusion and to minimize the effect of measurement errors on the tracking accuracy due to occlusion.
  • Our method can handle both 2D and 3D upper body tracking within the same framework. Using the DBN model, upper body tracking can be achieved through probabilistic inference over time.

Publication: Lei Zhang, Jixu Chen, Zhi Zeng, and Qiang Ji, 2D and 3D Upper Body Tracking with One Framework, International Conference on Pattern Recognition, 2008.PDF

Real-Time Action Recognition using HMM
In this work, we proposed a Hidden Markov Model for human action recognition in real-time.
  • We obtained the skeleton positions of human from Kinect depth camera and the builtin software. Based on the skeleton information, Hidden Markov Model is used to model the transition between the hidden states that define the action.

Gesture recognition for teaching mathematics