NVIDIA AI tool fills gaps in photos in a realistic way (video)

Researchers from NVIDIA present the results of a new deep learning algorithm that can reconstruct photos in a very realistic way. It's like a super-advanced version of Photoshop's Content Aware Fill, it can be used to reconstruct a corrupted image, or to remove content and fill in the resulting holes.

The method, which performs a process called “image inpainting”, could be implemented in photo editing software to remove unwanted content, while filling it with a realistic computer-generated alternative.

“Our model can robustly handle holes of any shape, size location, or distance from the image borders. Previous deep learning approaches have focused on rectangular regions located around the center of the image, and often rely on expensive post-processing,” the NVIDIA researchers stated in their research paper. “Further, our model gracefully handles holes of increasing size.”

To prepare to train their neural network, the team first generated 55,116 masks of random streaks and holes of arbitrary shapes and sizes for training. They also generated nearly 25,000 for testing. These were further categorized into six categories based on sizes relative to the input image, in order to improve reconstruction accuracy.

More details can be read at NVIDIA. Definitely check out the video below, the output is pretty amazing.