Nvidia uses artificial intelligence to fake realistic slow-motion videoJune 19, 2018
Creating slow motion footage is all about capturing a large number of frames per second. If you don’t record enough, then as soon as you slow down your video it becomes choppy and unwatchable. Unless, that is, you use artificial intelligence to imagine the extra frames.
New research from chip designer Nvidia does exactly that, using deep learning to turn 30 frames-per-second video into gorgeous, 240 frames-per-second slow-motion. Essentially, the AI system looks at two different frames and then creates intermediary footage by tracking the movement of objects from one frame to the next. It’s not the same as actually imagining footage like a human brain does, but it produces accurate (though not perfect) results.
The process will need refinement before it can be used commercially, but when improved, could be used to add slow-motion effects to smartphone footage after it’s been recorded.
“While it is possible to take 240-frame-per-second videos with a cell phone, recording everything at high frame rates is impractical, as it requires large memories and is power-intensive for mobile devices,” write the researchers in a paper describing their work on pre-print server arXiv. “For these reasons and others, it is of great interest to generate high-quality slow-motion video from existing videos.”
There are limitations though. For a start, the researchers had to train their deep learning system to handle specific types of footage. So, in order to slow down video of a car skidding through water they had to first train it on similar footage. This might limit how widely a consumer version of the research could apply slow-motion to target video. It’s also worth noting that such a technique can’t produce an infinite number of intermediary frames. Without data it becomes impossible for computers to predict the motion and behavior of real world objects. However, Nvidia’s system can produce up to seven intermediary frames — more than enough to create decent slow-motion.
Nvidia’s technique looks faster and easier than existing methods for faking slow-motion footage. Expect to see features like this appearing in your phone’s camera in the near future.