Fragments boost 3D TV
By
Eric Smalley,
Technology Research NewsIn January 2001, CBS spiced up its coverage of the Super Bowl with a special effect that allowed the broadcaster to freeze a replay, arbitrarily change the viewpoint and continue the replay. Researchers around the world are looking to take this technology further by enabling viewpoint changes as the action, including live-action, unfolds, and by letting viewers control the viewpoint.
The formidable technical challenge in presenting real-time, free-viewpoint three-dimensional video is the enormous amount of information contained in the stream of video information.
Researchers from Swiss Federal Institute of Technology (ETH) in Zürich have devised a way to process three-dimensional video in real-time that reduces the amount of data to the manageable level of 3 megabits per second.
The method promises to accelerate the development of three-dimensional television, three-dimensional tele-immersion systems, and programming that allows viewers to change viewpoints in real-time while watching televised events.
The researchers used the technique in their prototype videoconferencing system Blue-c. The three-dimensional video portal uses a pair of U-shaped booths that combine real-time image acquisition and projection. The booths' walls switch rapidly between opaque and transparent in synchronization with cameras and projectors in order to capture the user's image through the walls while allowing the user to see images projected on them.
The three-dimensional video processing technique is a "complete real-time free-viewpoint video system," said Stephan Würmlin, a researcher at the Swiss Federal Institute of Technology from.
The system could allow images to be rendered on different types of devices, said Würmlin. "The data representation allows for progressive transmission and display," he said. "This enables the technology to not only display images on projection walls but also [on] different devices, for example handhelds, smart phones or TV screens."
Key to the method is a way to issue updates to a three-dimensional image by inserting, deleting and updating just the changed portions of a video frame. These changes from multiple cameras are processed and merged into a single video stream. This way the three-dimensional geometry of the image does not have to be recalculated for every frame, which reduces computational load and network bandwidth consumption, said Würmlin.
The system calculates two-dimensional pixels from multiple cameras to determine the position, orientation and color of a set of irregular points in the three-dimensional space. The method represents these dynamic point samples, or three-dimensional fragments, independently of each other.
Traditional three-dimensional graphics build images out of triangles, each of which has to be connected to its neighbors. It is difficult to properly align the triangles of a three-dimensional image produced using multiple cameras, said Würmlin. Removing the need for connectivity information allows for more efficient image updating and data compression, he said.
Three-dimensional free-viewpoint video systems have been developed by researchers at Microsoft, the Max Planck Institute in Germany, the Massachusetts Institute of Technology and Carnegie Mellon University, said Würmlin. There is also a free-viewpoint television (FTV) project at Nagoya University in Japan. They all have limitations that make them impractical for real-time free-viewpoint three-dimensional video systems, Würmlin said.
These systems tend to need a lot of bandwidth -- some an order of magnitude more than the Swiss researchers' system, which needs an information flow of 3 megabits per second to provide decent image quality on a projection screen, said Würmlin.
Some of the three-dimensional video systems only work if the objects being captured on camera are modeled beforehand, and some use dense arrays of cameras -- as many as 100 -- in set positions, said Würmlin. "Our system is capable of capturing arbitrary and multiple objects [and] dealing with wide baselines -- currently two to three meters -- and arbitrary [camera] setups," he said.
Three-dimensional video fragments make for a more efficient system because only a subset of the cameras are used at any one time to capture three-dimensional information. In the researchers' prototype, any given frame uses ten of the cameras in the array to capture three-dimensional information and two to provide for color and texture information.
The system adjusts to network and system congestion by dynamically scaling back the number of active cameras, trading off image quality for system performance.
One challenge in using the system practically is that, though it reduces the required network bandwidth, the three-dimensional video stream is sensitive to network errors and needs to be made more resilient, according to Würmlin.
The method works with any real-time three-dimensional reconstruction method that extracts depth from images, said Würmlin. It also makes it easier to process special effects, he said.
The technology has been adopted for the next version of the Animation Framework Extension (AFX) of the Motion Picture Experts Group (MPEG) 4 standard for interactive and three-dimensional video, said Würmlin. The extension defines standards for representing and transmitting animations. "Thus, standardized streams and players are within reach," he said.
The method could be used in practical applications in one to three years, said Würmlin.
Würmlin's research colleagues were Edouard Lamboray and Markus Gross. The work appeared in the February, 2004 issue of Computers & Graphics. The research was funded by ETH Zürich and Mitsubishi Electronics Research Laboratory (MERL).
Timeline: 1-3 years
Funding: University
TRN Categories: Computer Vision and Image Processing; Human-Computer Interaction
Story Type: News
Related Elements: Technical paper, "3D Video fragments: Dynamic Point Samples for Real-time Free-Viewpoint and Video," Computers & Graphics, February, 2004.
Advertisements:
June 16/23, 2004
Page One
Fragments boost 3D TV
Internet ups power grid IQ
Fractals show machine intentions
VR tool re-creates hallucinations
Briefs:
Chip miniaturizes holography
Pressure adds depth to displays
Genes automate DNA machines
Scheme optimizes light chips
Silicon nanowires grown in place
Laser tweezer grabs varied specks
News:
Research News Roundup
Research Watch blog
Features:
View from the High Ground Q&A
How It Works
RSS Feeds:
News
| Blog
| Books 
Ad links:
Buy an ad link
| Advertisements:
|
 |
Ad links: Clear History
Buy an ad link
|
TRN
Newswire and Headline Feeds for Web sites
|
© Copyright Technology Research News, LLC 2000-2006. All rights reserved.
