Doctor of Philosophy
School of Electrical, Computer and Telecommunications Engineering
Zheng, Haoheng, Learning-based super-resolution with applications to intensity and depth images, Doctor of Philosophy thesis, School of Electrical, Computer and Telecommunications Engineering, University of Wollongong, 2014. https://ro.uow.edu.au/theses/4284
Resolution is an important attribute for evaluating the quality of an image. In most applications, it is desirable to have high-resolution images rather than lowresolution ones because people can appreciate more detailed information from high-resolutions. In addition, high-resolution images facilitate image analysis tasks performed by machines, such as scene recognition and classification. However, in practice high-resolution images are not always readily available for various reasons, such as the deficiency of the recording device or imaging far objects. Therefore, a super-resolution technique is usually required to enhance the resolution of the image.
The problem of super-resolution has already been extensively researched over two decades, during which different super-resolution algorithms have been developed. In this thesis, a thorough investigation of super-resolution methods is conducted. In addition to the traditional 2-D intensity image super-resolution methods, methods that enhance the resolution of 3-D depth images are also reviewed. Although super-resolution methods vary from each other, a category of methods, namely learning-based super-resolution, generally offers more accurate reconstruction results compared to other methods. In this thesis, three learning-based super-resolution methods are proposed.
The first super-resolution method solves the intensity image super-resolution problem. Traditional learning-based methods train a dictionary with a pre-defined size for reconstructing different kinds of HR patches; accurate HR images may not be able to recover from this trained dictionary due to the pre-defined size. To improve the reconstruction accuracy, a multi-dictionary sparse representation method is proposed, where the size of each dictionary is adaptive to the input patch. Specifically, the input image is first decomposed into patch level (small blocks). Then, for each input patch, a dictionary is generated from a image database containing high-resolution images; this dictionary contains selected atoms for recovering the high-frequency information from the input lowresolution patch. Compared to the conventional single-dictionary sparse representation method, the proposed method reconstructs high-resolution images with higher accuracy.
The second method reconstructs high-resolution depth images from their lowresolution counterparts. Different from intensity image super-resolution, depth image super-resolution has more resources that can be used for reconstructing more accurate high-resolution images. In this method, the high-resolution intensity image captured along with the low-resolution depth image is utilized for the depth reconstruction. To be more specific, a joint regularization constraint is constructed from the high-resolution intensity image first. This constraint is then incorporated into the proposed multi-dictionary sparse representation method to regulate the high-resolution reconstruction. In addition, an adaptive method for estimating the joint regularization parameter is proposed. The experimental results demonstrate the effectiveness of the proposed regularization constraint as well as the estimated regularization parameter.
The last method enhances the depth resolution by formulating a joint regularization problem, where the target image and its sparse code are computed simultaneously. Similar to the second proposed method, different constraints are built from different high-frequency sources to improve the reconstruction accuracy. The sparse reconstruction constraint is constructed by exploiting high-frequency information from the external image database, whereas the local and non-local constraints are formed by extracting information from the internal low-resolution input image and the registered high-resolution intensity image. Furthermore, an adaptive method for estimating the local regularization parameter is proposed. The proposed method is able to generate sharper high-resolution depth images with fewer artifacts compared with other state-of-the-art super-resolution approaches.