Searching and describing human motion

The amount of media being uploaded to the Internet is growing at an incredible rate. As an illustration, approximately 75 hours of video are uploaded to Youtube each minute, where approximately 30% of the videos contain human motion such as sport or music video. Consequently, new techniques and methods to search and describe contents related to human motion are sorely needed, since current search techniques mainly depend on user-supplied tags, which are often ambiguous and subjective when those tags are used to describe human motion. For example, a video containing “John Doe running and jumping into a lake” can be tagged as “John Doe”, “lake”, “running and jumping”, “funny video”, etc. Being able to search for a specific motion has many applications. For example, searching for a specific movement in a sport in order to improve a person’s sporting performance by comparing to that of a professional athlete’s using automatically extracted movement features (such as a famous golfer’s swing, a famous tennis player’s forehand, etc.). This scenario will be possible if a method to objectively describe human motion existed. Searching human motion would be as natural as recording a motion and using it as yet another search term without having to think about the subjectivity of user-supplied tags and how someone else would “describe” that motion. To achieve this, three things are required: a new multimedia communication format (since currently popular search techniques predominantly use simple text terms), a new human motion description language (since an objective and consistent method to describe human motion is also required), and feature extraction and matching technique for human motion search applications. To communicate advanced multimedia queries, Multimedia Query Format (MQF) is presented in this thesis. MQF is a communication format for a structured multimedia search that goes beyond current text-based search currently in popular use. Instead of restricting itself to