Advanced Digital Sciences Center Research Scientist Jiangbo Lu and his video cutout technology, CuteChat, was one of five demonstrations to win the DEMOguru award at the DEMO Asia 2012 conference earlier this month in Singapore.

The conference, which ran from February 29 to March 2, is the first to be held in Asia and is a launchpad for emerging technology and trends. DEMO conferences are held across the world and have earned a reputation for consistently identifying cutting-edge technology of the future.

Pixtelz co-founder Boon Leng Lee speaks with ADSC’s Jeremy Heng via Skype during the live demonstration of CuteChat at the 2012 DEMO Asia conference

Lu developed the video cutout technology at ADSC as part of University of Illinois at Urbana-Champaign Electrical and Computer Engineering Associate Professor Minh Do’s research project on low-cost virtual reality. CuteChat is a lightweight video chat system that allows the average user to manipulate the background of their video. CuteChat removes the background for the user’s camera feed and replaces it with another background, such as a landscape scene, or video feed.

Lu and his company, Pixtelz, were hand selected to participate in the inaugural DEMO Asia, as part of 76 teams from 14 countries.

From left, ADSC researcher and Pixtelz co-founder Jiangbo Lu, ADSC Deputy Director Jeremy Heng, and Pixtelz co-founder Boon Leng Lee staffed Pixtelz’s CuteChat booth at the 2012 DEMO Asia conference

“It’s a great recognition for Pixtelz, a less-than-6-month old, tiny startup, and is another milestone for us,” Lu said.

During the demonstration, Boon Leng Lee, Pixtelz CEO and co-founder, had a live conversation via Skype with ADSC Deputy Director Jeremy Heng and ADSC software engineer Huanliang Sun. By using prerecorded footage of locations around Singapore, Heng was able to fool Lee into believing that he was waiting for a flight at an airport, eating lunch in a food court, working in his office and even showed him photos from a recent vacation. He also demonstrated some of the special effects CuteChat offers, such as fuzzing out confidential information, hiding people in the background of a room, changing the foreground of the image and displaying presentation materials.

“This was another crowd-pleaser, as the entrepreneur showed off how he could fake out his boss, or even his wife, by pretending to be in his office, while he was really out having fun,” Executive Producer of DEMO Matt Marshall said in his DEMO Asia review on VentureBeat.

Pixtelz co-founders Jiangbo Lu (left) and Boon Leng Lee (right) were awarded one of five DEMOguru awards for their CuteChat technology at the 2012 DEMO Asia conference

Lu received positive feedback from panel judges, senior management and Singapore’s Agency for Science, Technology and Research (A*STAR) colleagues that the demonstration was “fantastic” and one of the “best demos” of the conference. Additionally, he said the demonstration looked so real, many audience members weren’t able to figure out where Heng was, even after the demonstration. Heng and Sun were actually in a meeting room in the same building as the conference.

“I think it really shows that something developed in the lab at ADSC, as a result of basic research, can step out of the lab and be well liked by the users,” Lu said. “This is a very good example of the excellence of our research and also its relevance to the society and its potential commercial value.”

In addition to the special effects CuteChat adds to video chatting, CuteChat’s object-based video coding reduces bandwidth requirements and speeds up the video coding process, both by a factor of four. This video cutout technology was the first ADSC research output to be protected as a trade secret by Exploit Technologies, the commercialization arm of A*STAR.

“For Pixtelz, such an award brings much attention and publicity,” Lu said. “We now have good networking with participants from various backgrounds offline. It makes our next move easier.”

As for the future of Pixtelz, Lu said they plan to release their QuickToon video stylization software for free trial download in the coming months. They also plan to seek further funding and hope to leverage this award to attract more potential users worldwide.

For more information about CuteChat or to view a video demonstration, visit ADSC’s Research Highlights page or YouTube page.

The Advanced Digital Sciences Center is a Singapore-based research center for faculty of the College of Engineering at the University of Illinois at Urbana-Champaign. ADSC focuses on breakthrough innovations in information technology.

Bingbing Ni, postdoctoral fellow at the Advanced Digital Sciences Center, was presented the Best Paper Award at the 2011 Pacific-Rim Conference on Multimedia (PCM 2011) for his research on image re-emotionalizing.

PCM is a major annual conference that highlights the advances and research results in the fields of theoretical, experimental and applied multimedia analysis and processing. From over 175 paper submissions, the conference awards one best paper award and one best student paper award. In the winning paper, “Image Re-Emotionalizing,” Ni and his collaborators developed a system for synthesizing user-specified emotional affection onto any input image.

Ni works with University of Illinois at Urbana-Champaign professor Pierre Moulin on ADSC’s Interactive Digital Media project, Multi-Modal Visual Analytics. His research focuses on computer vision, multimedia computing and machine learning.

According to Ni, the algorithm could have many potential interesting multimedia applications. If a user wants to change the coloring or shading of an image so that viewers perceive the photo as being sad or frightening or awe-inspiring, Ni’s algorithm would colorize and shade the image in such a way that elicits this reaction. This would be especially useful to an inexperienced user who would have little knowledge about how to alter photos in programs, such as Adobe Photoshop.

“The result of this research is an algorithm whose input is any arbitrary image and a user-specific emotion category, such as amusement, neutral, sadness or fear, and the output is a synthesized image with that target emotion,” Ni said. “I think this is the world’s first work on synthesizing emotion information onto arbitrary input images. Winning this award is regarded as the acknowledgement from the community for this promising and brand-new research direction.”

Ni co-authored the paper with National University of Singapore (NUS) Electrical and Computer Engineering Ph.D. student Mengdi Xu, NUS Electrical and Computer Engineering Professor Shuicheng Yan and Nanjing University of Science and Technology (China) Computer Science Professor Jinhui Tang.

The research Ni has done on image re-emotionalizing ties closely with many Interactive Digital Media projects at ADSC. For example, Ni’s image-based rendering can be incorporated into University of Illinois at Urbana-Champaign Electrical and Computer Engineering Associate Professor Minh Do‘s Immersive Telepresence for Entertainment and Meetings research project. In this project, Do aims to present videos in different styles, such as cartoon, oil painting or as a portrait. Ni’s algorithm now allows them the option of presenting videos and images with different emotions.

The Advanced Digital Sciences Center is a Singapore-based research center for faculty of the College of Engineering at the University of Illinois at Urbana-Champaign. ADSC focuses on breakthrough innovations in information technology.

Researchers at the Advanced Digital Sciences Center specializing in computer vision technologies are developing an intuitive, reliable and fast solution for identifying and retrieving relevant information about pill medications. Applications for the pill identification technology include use in hospital and nursing home pill packing machines, as well as a mobile device for patients, specifically the elderly, to ensure the right pills are being taken each day.

In hospitals, pills are automatically packaged by machines, but mistakes often lead to manual quality control. ADSC’s algorithm would improve pill identification accuracy, ensuring that the correct patient receives the correct medication each day.

Many of today’s hospitals currently have pill packing machines, which automatically package the pills for each patient. Some of these machines include quality control modules to check for physical defects of pills, gaps in package seal, correct and readable labels and quantity errors, using technology such as barcodes and radio-frequency identification (RFID). However, in many machines, these techniques are inaccurate or outdated, often leading to time-consuming manual quality control.

ADSC‘s pill identification research is led by Gang Wang, a research scientist at ADSC and University of Illinois at Urbana-Champaign computer science graduate. It was inspired by a conversation about personal pill containers between ADSC Director Marianne Winslett and ADSC research scientist Jiangbo Lu, who helped initiate the research.

According to Wang, ADSC’s technology works directly on pill images using image processing and computer vision techniques. First, given an input image, the pill part is automatically separated from the background. Then different techniques are applied to detect and represent key visual features of the pill, including major colors, shape and imprint. Finally, a two-step image matching approach is performed to retrieve the top matched pills from an online image database, based on the similarity of visual features.

ADSC’s pill identification algorithm complements pill packing machines, such as the one shown here, by automating the inspection of pill packets.

Wang and his team have been collaborating with Singapore’s National University Hospital to develop a final product in a testing lab. Wang is using the algorithm created at ADSC to complement the current pill packing machines the hospital already has. The hospital’s machines have been programmed with the new algorithm, so the machines are able to automatically detect anything incorrect in the packing.

In developing the algorithm, Wang and his team faced many challenges, such as low quality images of pills, due to reflections on bag surface, and pills with similar shapes, textures or colors. Additionally, when there are multiple pills in a bag, some may be occluded or identical pills may have different orientations.

“We have addressed the imaging and orientation issues, but we are still working on how to tell small differences between the pills, or the verification accuracy,” Wang said.

Ideally, the team believes that every hospital in the world would benefit from this technology. In addition, they plan to also push the mobile version of their pill identification technology, as it will most benefit the aging population.

“Aging has become a very big problem, so we’re seeing more and more senior citizens staying home without other relatives or healthcare providers,” Wang said. “They have to take care of themselves and often have to take a lot of pills every day. Our staff can integrate our algorithm with the proper hardware and we can help them make sure they take the right pills each day.”

Wang and his team plan to commercialize the application for use in hospitals to complement pill packaging machines in the near future. They are also looking to create a prototype this summer and expect to integrate the algorithm with the hardware later in 2012.

The Advanced Digital Sciences Center is a Singapore-based research center for faculty of the College of Engineering at the University of Illinois at Urbana-Champaign. ADSC focuses on breakthrough innovations in information technology.

The human ear is an intricate body part allowing humans to hear and identify where sounds originate. It allows humans to have a sense of where they are in the world.

University of Illinois at Urbana-Champaign Electrical and Computer Engineering Professor Doug Jones leads a team of researchers working to digitally recreate this natural phenomenon by developing low-cost, realistic, real-time audio-visual telepresence at the Advanced Digital Sciences Center in Singapore.

Jones’ team includes ADSC researcher Shengkui Zhao and ADSC software engineerSaima [dot] a

adsc [dot] com [dot] sg ( Saima Ahmed). Their research project, Realistic Audio Telepresencing for Entertainment and Meetings (RATEM), aims to create a system that provides a full 4D audio-visual remote reality, using three spatial dimensions plus time. A system like this could revolutionize teleconferencing, augmented reality and gaming by providing a true 4D sound and video perception. Additionally, it would enable the ability to create a telepresence where participants are placed in a virtual setting, but interact as though they are physically present together.

“You would be able to talk to the person on your right or left and hear them and look at them in the right place and carry on simultaneous conversations,” Jones said of these virtual meetings.

Jones’ research group is collaborating with University of Illinois at Urbana-Champaign Electrical and Computer Engineering Associate Professor Minh Do’s Immersive Telepresence for Entertainment and Meetings (ITEM) research group at ADSC. While Do focuses on the vision portion of the research, Jones is working to recreate sounds accurately and from the correct direction, leading to a 4D audio experience.

According to Jones, it is easy for humans to tell where a sound is coming from, but it is much more difficult for a computer, which would be used during teleconferencing. The human ear can locate sounds from above and below, as well as in front or behind and to the left and right. By using the brain, external ear and internal ear, humans are able to estimate the location of a sound. However, it is difficult to replicate this on a computer as scientists don’t fully understand the human body’s biological neural determination process.

To solve this problem, most scientists are using microphone array signal processing to localize sound sources. Unfortunately, current methods require large arrays of many microphones and expensive, special-purpose computers to do so in real time.

ADSC’s actual microphone array, which measures just a few millimeters. University of Illinois at Urbana Champaign Electrical and Computer Engineering Professor Doug Jones developed the microphone array at Illinois in 2005.

“With those kinds of installations, you can do what we’re talking about,” Jones said. “But it costs several hundred thousand dollars per room. We want to do it at a reasonable price.”

Jones and Zhao aim to solve this fundamental research challenge by perfecting 3D audio direction-finding, as well as directional voice acquisition and binaural 3D audio reconstruction.

The pair recently developed a real-time 3D direction finding algorithm for a microphone array. Using a microphone array and an ordinary laptop, the program is able to track the location of a speaker’s voice. According to Jones, the new hardware and software are smaller, use less power, are less expensive and are more accurate than today’s conventional approaches that use much larger microphone arrays.

In 2005, Jones developed the smallest microphone array in the world at the University of Illinois. The array has four microphones that each record sounds from a different direction. He originally developed the array to be used in hearing aids, but it is now being further developed at ADSC to discover the relative strength of a sound at the different microphones to determine what direction the sound came from.

When a human hears a sound, their ears split the sound up into different frequency bands to determine where the sound in that frequency range is coming from. Additionally, ears are 15 centimeters apart, meaning a sound reaches one ear before the other, causing a time delay. This helps the ear determine where the sound is coming from. Using Jones’ biologically-inspired microphone array, with each microphone measuring only a few millimeters and located within a few millimeters of each other, the researchers are able to locate the direction of sounds. Since the microphones are so close together, the array doesn’t use time delay, but rather the directional pattern of the sound to determine the sound source. Jones and his team are attempting to identify the sound source with just a few microphones, or essentially the way the brain does it.

Researcher Shengkui Zhao and ADSC Director Marianne Winslett test ADSC’s microphone array to track where a speaker’s voice is originating.

Jones and Zhao aim to make their 4D audio technology practical, small and affordable, in hopes that it will be usable in many different areas.

While Jones began developing theoretical techniques at Illinois, he said at that time there wasn’t interest in taking the next step toward practical research. By bringing the project to ADSC, he has been able to work on building better algorithms and optimizing implementations.

“I don’t think we would’ve been able to get where we are today if we weren’t doing the research in Singapore,” Jones said. “It has been great to work with the computing and video researchers at ADSC to try to integrate our research with other modalities, like vision, to combine sound with images and video.”

As Jones and Zhao have succeeded in tracking the directions of sounds with a microphone array, their research will now shift to focusing on finding a solution for the cocktail party effect, or separating voices and sounds from other background noises, and realistically reconstructing a sound.

Jones and his team had two research papers accepted to the 2012 Design, Automation and Test in Europe Conference (DATE) and the 2012 Conference on Industrial, Electronics and Applications (ICIEA) and have submitted three other papers to various international conferences. Additionally, Zhao is close to finishing development of a real time 3D sound reconstruction system, and the team is beginning to research other possible applications for the findings, such as defense, virtual sports or possibly robotics.

“I think this research definitely has market potential,” Jones said. “My dream would be that every video camera or cell phone would be able to record 3D sound and listen to the recording in full 3D. Microphones are cheap and arrays are small, so this technology could be on even the smallest cell phone or video camera. Technology is almost at that point.”

The Advanced Digital Sciences Center is a University of Illinois at Urbana-Champaign research center based in Singapore, led by Illinois Computer Science and Electrical and Computer Engineering faculty. ADSC focuses on breakthrough innovations information technology.

ADSC researchers took the European Conference on Computer Vision (ECCV) by a storm last month in Firenze, Italy. ADSC had seven papers accepted and presented at the main conference, in addition to presenting papers during two workshops. ECCV is one of the top conferences for researchers in computer vision.

“ADSC’s strong presence at ECCV with 7 conference and 2 workshop papers is on par with the likes of UC Berkeley or Stanford, as well as the University of Illinois at Urbana-Champaign, firmly establishing ADSC as a center for world-leading computer vision research,” ADSC’s Interactive Digital Media Program Director Stefan Winkler said.

ADSC researcher Tianzhu Zhang‘s presented his paper, “Low-Rank Sparse Learning for Robust Visual Tracking,” at the October 2012 European Conference on Computer Vision (ECCV) in Firenze, Italy. Six other ADSC conference papers were accepted to ECCV, along with two workshop papers.

Included in the paper submissions were:

  • Qiang Zhou and Gang Wang, “Learning to Recognize Unsuccessful Activities Using a Two-Layer Latent Structural Model.”
  • Bingbing Ni, Pierre Moulin, Shuicheng Yan, “Order-Preserving Sparse Coding for Sequence Classification.”
  • Tianzhu Zhang, Bernard Ghanem, Si Liu, Narendra Ahuja, “Low-Rank Sparse Learning for Robust Visual Tracking.”
  • Kui Jia, Tsung-Han Chan, Yi Ma, “Robust and Practical Face Recognition via Structured Sparsity.”
  • Tsung-Han Chan, Kui Jia, Eliot Wycoff, Chong-Yung Chi, Yi Ma, “Towards Optimal Design of Time and Color Multiplexing Codes.”
  • Zinan Zeng, Tsung-Han Chan, Kui Jia, Dong Xu, “Finding Correspondence from Multiple Images via Sparse and Low-Rank Decomposition.”
  • Zhengxiang Wang, Shenghua Gao, Liang-Tien Chia, “Learning Class-to-Image Distance via Large Margin and L1-Norm Regularization.”

Additionally, ADSC researcher Qiang Zhou presented two papers at two of the 21 workshops that were held before and after the conference.

  • Jiwen Lu and Gang Wang, “Human-Centric Indoor Environment Modeling from Depth Videos,” 2nd Workshop on Consumer Depth Camera for Computer Vision, in conjunction with ECCV 2012.
  • Qiang Zhou and Gang Wang, “Atomic Action Features: A New Feature for Action Recognition,” 4th Workshop on Video Event Categorization, Tagging and Retrieval, in conjunction with ECCV 2012.

TheAdvanced Digital Sciences Centeris aUniversity of Illinois at Urbana-Champaignresearch center in Singapore. It is led by Electrical and Computer Engineering and Computer Science faculty at the University of Illinois. ADSC focuses on innovations in information technology.

University of Illinois Advanced Digital Sciences Center (ADSC) researcher Jiwen Lu was recently awarded the Best Student Paper Award from the Pattern Recognition and Machine Intelligence Association (PREMIA) of Singapore for a paper published at the 2011 International Conference on Computer Vision (ICCV).

ICCV is held once every two years and is a highly selective computer vision conference. PREMIA is a member of the International Association for Pattern Recognition.

The paper, “Discriminative Multi-manifold Analysis for Face Recognition from a Single Training Sample per Person,” addresses a challenging problem in the face recognition area and was co-authored with the Associate Professor Tan Yap-Peng in Nanyang Technological University (NTU) and University of Illinois ADSC Research Scientist Gang Wang.

“We proposed a novel discriminative multi-manifold analysis to solve the problem of face recognition from a single training sample per person,” Lu said. “Most face recognition methods don’t work well or even fail to work when there is only one training sample per person in the database. Our purposed method can effectively solve this problem under this scenario.”

Additionally, a longer version of the proposed research method was published in the IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), a top journal in computer vision.

Lu works with University of Illinois at Urbana-Champaign Professor Thomas Huang’s Understanding the Built Environment by Observing Human Activities research group at ADSC in Singapore. His research interests include computer vision, pattern recognition, machine learning and biometrics.

The Advanced Digital Sciences Center is a University of Illinois at Urbana-Champaign research center in Singapore. It is led by Electrical and Computer Engineering and Computer Science faculty at the University of Illinois. ADSC focuses on breakthrough innovations in information technology.