Super-Resolution Using Unsupervised Deep Internal Network

The Need

Signal enhancement of corrupted input collected under suboptimal conditions, poses multiple challenges, including super-resolution, denoising, deblurring, dehazing and lens artifact correction. While deep-learning techniques have dramatically improved super-resolution performance, they remain limited to specific, high-resolution training data, free of distracting artifacts. For single-image enhancement that does not obey ideal conditions, these limitations demand lengthy and exhaustive training on external databases and strict fulfillment of the training conditions to obtain satisfactory results.

The Solution

This invention introduces zero-shot super-resolution (ZSSR), which exploits deep-learning methodologies, without requiring any prior image or training data. The internal recurrence of data within the single input image is exploited for on-line training of a small image-specific convolutional neural network (CNN) in examples extracted from the lowresolution image itself. The method bears no recurrence patch-size limitation, enabling adaptation of the CNN to different settings for the same image, and requires no external information or prior training. This unsupervised CNNbased super-resolution method has been shown to substantially outperform externally trained state-of-the-art superresolution of suboptimal, low-resolution images (Figure A). In cases of ideal imaging conditions, the ZSSR output proved competitive to that of state-of-the-art supervised methods (Figure B). A) Super-resolution of a suboptimal low-resolution image using the ZSSR approach versus state-of-the-art (SotA)
approaches. B) Super-resolution using ZSSR as compared to state-of-the-art methods, of a low-resolution image generated
under ideal, supervised conditions.

Applications and Advantages

Advantages

• Suitable for a wide range of images and data types
• Applicable for images of any size and any aspect ratio

No pre-training requirement

Adaptable to images with known or unknown imagining conditions

Time-effective training

• No requirement of side information/attributes
• No requirement of additional images

Applications

This invention can be applied in a range of images and distortion types requiring enhancement, including:

Single video sequences

Old photos

Noisy images

Biological data

Medical images (e.g., fMRI)

Audio sequences

Development Status

The image-enhancement platform has been demonstrated effective for super-resolution, dehazing, watermark removal and image defect elimination (link to project web-page [1]). Ongoing work is optimizing the technology for fMRI image applications. A patent application has been submitted.

Market Opportunity

This innovation can be of commercial relevance in a wide range of markets reliant upon imaging technologies, including:
Consumer image processing software programs

Military and security

Medical imaging
Microscopy and other lab imaging