저해상도를 고해상도로 바꿔주는 AI

https://techxplore.com/news/2020-06-artificial-intelligence-blurry-sharper.html

 

The system cannot be used to identify people, the researchers say: It won't turn an out-of-focus, unrecognizable photo from a security camera into a crystal clear image of a real person. Rather, it is capable of generating new faces that don't exist, but look plausibly real.

While the researchers focused on faces as a proof of concept, the same technique could in theory take low-res shots of almost anything and create sharp, realistic-looking pictures, with applications ranging from medicine and microscopy to astronomy and satellite imagery

Traditional approaches take a low-resolution image and 'guess' what extra pixels are needed by trying to get them to match, on average, with corresponding pixels in high-resolution images the computer has seen before. As a result of this averaging, textured areas in hair and skin that might not line up perfectly from one pixel to the next end up looking fuzzy and indistinct.

The Duke team came up with a different approach. Instead of taking a low-resolution image and slowly adding new detail, the system scours AI-generated examples of high-resolution faces, searching for ones that look as much as possible like the input image when shrunk down to the same size.

The team used a tool in machine learning called a "generative adversarial network," or GAN, which are two neural networks trained on the same data set of photos. One network comes up with AI-created human faces that mimic the ones it was trained on, while the other takes this output and decides if it is convincing enough to be mistaken for the real thing. The first network gets better and better with experience, until the second network can't tell the difference.

 

논문을 봤는데 구조는 안나와있고, 그림이 나와있다. latent space 를 그래픽으로 보여줬는데 이해하기 힘들다. 

다시 선형대수를 보며 읽어봐야겠다.

 

reference

PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models, arXiv:2003.03808 [cs.CV] arxiv.org/abs/2003.03808