Advanced Digital Sciences Center researchers Bingbing Ni and pei [dot] yong adsc [dot] com [dot] sg (Yong Pei) won the 2012 Human Activities Recognition and Localization (HARL) competition, held in conjunction with International Conference on Pattern Recognition (ICPR), the leading conference on pattern recognition. Ni and Pei will present their winning method at ICPR in Tsukuba, Japan next month.

The HARL competition invited participants to submit algorithms to automatically detect, classify and localize activities taking place in Kinect video. This emerging video modality outputs both color and depth image sequences, the latter showing the distance of each point in a scene to the camera. HARL is the world’s first competition that focuses on recognizing and localizing complex human activities in footage from a Kinect camera.

ADSC Researchers Bingbing Ni, left, and Yong Pei, right, won the 2012 Human Activities Recognition and Localization competition in September. They will present their winning method during the November 2012 International Conference on Pattern Recognition (ICPR).

HARL participants were given benchmark datasets of video clips showing complex activities taking place in a typical office setting. Participants created algorithms that could analyze a long video clip and precisely determine when and where the activity took place in the video.

“Previous activity recognition research focused primarily on classification, which means determining what the human is doing,” Ni said. “But in HARL’s more realistic and challenging setting, we had to determine when, where and what the humans were doing. Knowing when and where is more important and difficult than knowing what.”

According to Ni, by winning this award, the team has demonstrated that their algorithm and method for detecting activities is state-of-the-art and is the best option for recognizing actions in Kinect footage.

“Our team’s method is based on fusing the depth image with the grayscale image in every step of the processing pipeline, including human key pose extraction, object detection, interaction contextual modeling and a Bayesian network based action detector,” Ni said.

Ni and Pei developed algorithms to detect, classify and localize activities in Kinect video. They used their algorithms to identify actions such as a discussion between two people, see left, handshaking or talking on the phone.

Of the teams submitting results, Ni and Pei ranked first in both classification and localization of activities. Visit HARL for more information about the competition results.

Ni and Pei work with University of Illinois at Urbana-Champaign Electrical and Computer Engineering Professor Pierre Moulin on the Multi-modal Visual Analytics research project at ADSC. The project aims to develop methods for detection, recognition and localization of events and actions in depth video. This research has the potential to significantly impact video-based rehabilitation and sports training, video surveillance and many other areas.

TheAdvanced Digital Sciences Centeris aUniversity of Illinois at Urbana-Champaignresearch center in Singapore. It is led by Electrical and Computer Engineering and Computer Science faculty at the University of Illinois. ADSC focuses on breakthrough innovations in information technology.

The Advanced Digital Sciences Center (ADSC) and A*STAR’s Institute for Infocomm Research (I2R) hosted a Summer School on Multimodal Signal Analysis from June 28 – July 4, 2012. The goal of the school was to expose attendees to the different modalities of signal processing and analysis, such as image, 2D and 3D video, brain signals and social media, as well as their respective tools.

“We covered the currently hot research areas and latest happening technologies, such as 3D video processing and applications that were covered by three different lecturers at different angles,” said Viet-Anh Nguyen, ADSC Researcher and Summer School presenter. “They are not overlapping, but complementary to each other and gave the students in-depth knowledge about this area.”

Summer School 2012

Over 30 master’s and Ph.D. level students and researchers from local universities, research institutes and industry attended the school, including attendees from DSO National Laboratories, Singapore’s Ministry of Home Affairs and Panasonic Singapore Laboratories.

“The goal of the school was to get to know the other people attending the courses and exchange ideas on the wide range of topics we covered, such as video, brain signals or audio processing,” ADSC’s Interactive Digital Media Program Director Stefan Winkler said.

The program included morning and afternoon tutorial lectures by faculty members from the University of Illinois at Urbana-Champaign, National University of Singapore, Yonsei University, Singapore University of Technology and Design and researchers from ADSC, I2R and Microsoft Research Asia (MSRA). In the middle of the program, a social event was held on July 2 at Labrador Nature Reserve, providing a more relaxed setting for lecturers and participants to mingle. The program ended with a poster and demo session on July 4, where local researchers and students presented their work.

“The summer school was a success because of the distinguished speakers we brought in, such as University of Illinois at Urbana-Champaign Electrical and Computer Engineering Associate Professor Minh Do and University of Illinois at Urbana-Champaign Electrical and Computer Engineering Adjunct Professor Yi Ma, as well as the high quality talks,” said Jiangbo Lu, ADSC Research Scientist and Summer School presenter. “We hope the students came away from the sessions with more information about cutting-edge research areas, problems, theories, application and trends.”

Poster Session at Summer School 2012

The Advanced Digital Sciences Center is a University of Illinois at Urbana-Champaign research center in Singapore. It is led by Electrical and Computer Engineering and Computer Science faculty at the University of Illinois. ADSC focuses on breakthrough innovations in information technology.

Writer: Katie Carr, ADSC; Photos: Alex Chong, ADSC, of a lecture by ADSC’s Interactive Digital Media Program Director Stefan Winkler and a poster session discussion between students and ADSC’s sports application analyst shaunak [dot] ahuja