Breakthrough lets you touch videos instead of just watch

(Image: Abe Davis/MIT CSAIL)

Watching a video is a fairly passive experience -- the information appears before your eyes and you take it in. But researchers from the Massachusetts Institute of Technology's Computer Science and Artificial Intelligence Laboratory (CSAIL) have come up with a new imaging technique that could make videos far more interactive.

The team calls the new breakthrough "interactive dynamic video." It uses cameras to to capture even the most minuscule vibrations and ties in with algorithms to predict how an object moves. A video demonstration shows a simulation of a bush that was recorded when blowing in a breeze. Following the recording, the MIT team could manipulate the bush themselves by touching it.

Perhaps the biggest use is in special effects for movies and in augmented reality applications. CSAIL suggests this could reduce the use of green screen in movies, which is great, because practical sets are always better. (Look how the original "Star Wars" holds up). CGI characters are often put in front of real-life objects, and this could make them interact in a way that looks more true to life.

Researcher and PhD student Abe Davis calls out Pokémon Go as an AR game that could be improved by dynamic video. If, say, a Spearow appears on a pile of leaves, the leaves don't rustle or move. With interactive dynamic video, the Spearow could interact with the leaves rather than simply sit on top of them. A video shows mock ups of a Caterpie crawling out of a bush and a Pikachu moving around a jungle gym.

There's likely a long way to go before we see this type of technology hit mainstream applications, but it could make games, movies and videos far more interactive than they currently are.

via MIT, TechCrunch