(45-5) 08 * << * >> * Russian * English * Content * All Issues

A method for optimal linear super-resolution image restoration
A.I. Maksimov 1, V.V. Sergeyev 1,2

1 Samara National Research University, Moskovskoye Shosse 34, 443086, Samara, Russia,
2 IPSI RAS – Branch of the FSRC "Crystallography and Photonics" RAS,
443001, Samara, Russia, Molodogvardeyskaya 151

 PDF, 1731 kB

DOI: 10.18287/2412-6179-CO-909

Pages: 692-701.

Full text of article: Russian language.

Abstract:
In this paper, we propose a super-resolution (pixel grid refinement) method for digital images. It is based on the linear filtering of a zero-padded discrete signal. We introduce a continuous-discrete observation model to create a reconstruction system. The proposed observation model is typical of real-world imaging systems - an initially continuous signal first undergoes linear (dynamic) distortions and then is subjected to sampling and the effect of additive noise. The proposed method is optimal in the sense of mean square recovery error minimization. In the theoretical part of the article, a general scheme of the linear super-resolution of the signal is presented and expressions for the impulse and frequency responses of the optimal reconstruction system are derived. An expression for the error of such restoration is also derived. For the sake of brevity, the entire description is presented for one-dimensional signals, but the obtained results can easily be generalized for the case of two-dimensional images. The experimental section of the paper is devoted to the analysis of the super-resolution reconstruction error depending on the parameters of the observation model. The significant superiority of the proposed method in terms of the reconstruction error is demonstrated in comparison with linear interpolation, which is usually used to refine the grid of image pixels.

Keywords:
digital images, super-resolution, continuous-discrete observation model, linear system, optimal recovery, impulse response, frequency response, optimal reconstruction error.

Citation:
Maksimov AI, Sergeyev VV. A method for optimal linear super-resolution image restoration. Computer Optics 2021; 45(5): 692-701. DOI: 10.18287/2412-6179-CO-909.

Acknowledgements:
The work was partly funded by the Russian Foundation for Basic Research under project No 19-31-90113 (“Introduction”, “General method of signal linear super-resolution”, “Continuous-discrete observation model”, “Optimal restoration of discrete values of a continuous signal”, “Optimal restoration of discrete values of a continuous signal – frequency domain analysis”, “Error of the optimal restoration” and  “Optimal restoration of a whole continuous signal”) and research project No 19-07-00474 (“Experimental research of the proposed method”).

References:
  1. Isaac JS, Kulkarni R. Super resolution techniques for medical image processing. International Conference on Technologies for Sustainable Development (ICTSD) 2015: 7095900. DOI: 10.1109/ICTSD.2015.7095900.
  2. Sano Y, Mori T, Goto T, Hirano S, Funahashi K. Super-resolution method and its application to medical image processing. 2017 IEEE 6th Global Conference on Consumer Electronics (GCCE) 2017: 1-2. DOI: 10.1109/GCCE.2017.8229301.
  3. Chainais P, Pfennig P, Leray A.Quantitative control of the error bounds of a fast super-resolution technique for microscopy and astronomy. 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2014:  2853-2857. DOI: 10.1109/ICASSP.2014.6854121.
  4. Shen H, Ng MK, Li P, Zhang L. Super-resolution reconstruction algorithm to MODIS remote sensing images. Comput J 2009; 52(1): 90-100. DOI: 10.1093/comjnl/bxm028.
  5. Shi F. Yuan J, Zhu X. Fast super-resolution reconstruction for video-based pattern recognition. Fourth International Conference on Natural Computation 2008; 4: 135-139. DOI: 10.1109/ICNC.2008.553.
  6. Zamani NA, Darus MZA, Abdullah SNH, Nordin MJ. Multiple-frames super-resolution for closed circuit television forensics. International Conference on Pattern Analysis and Intelligent Robotics 2011; 1: 36-40. DOI: 10.1109/ICPAIR.2011.5976908.
  7. Kim SP, Bose NK. Reconstruction of 2-D bandlimited discrete signals from nonuniform samples. IEE Proc F Radar Signal Process 1990; 137(3): 197-204. DOI: 10.1049/ip-f-2.1990.0030.
  8. Bose NK, Lertrattanapanich S, Chappali MB. Super-resolution with second generation wavelets. Signal Process Image Commun 2004; 19: 387-391. DOI: 10.1016/j.image.2004.02.001.
  9. Stark H, Oskoui P. High resolution image recovery from image-plane arrays, using convex projections. J Opt Soc Am A 1989; 6: 1715-1726. DOI: 10.1364/JOSAA.6.001715.
  10. Elad M, Feuer A. Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Trans Image Process 1997; 6(12): 1646-1658. DOI: 10.1109/83.650118.
  11. Segall CA, Molina R, Katsaggelos AK, Mateos J. Bayesian high-resolution reconstruction of low-resolution compressed video. Proc IEEE Int Conf on Image Processing 2001; 2: 25-28. DOI: 10.1109/ICIP.2001.958415.
  12. Tsai RY, Huang TS Multiple frame image restoration and registration. In Book: Advances in computer vision and image processing. Greenwich, CT: JAI Press Inc; 1984: 317-339.
  13. Bevilacqua M, Roumy A, Guillemot C, Alberi-Morel ML. Low-complexity single-image super-resolution based on non-negative neighbor embedding. Proceedings British Machine Vision Conference 2012: 135. DOI: 10.5244/C.26.135.
  14. Timofte R, De Smet V, Van Gool L. A+: Adjusted anchored neighborhood regression for fast super-resolution. In Book: Cremers D, Reid I, Saito H, Yang M-H, eds. Computer vision – ACCV 2014. New York: Springer; 2015: 111-126. DOI: 10.1007/978-3-319-16817-3_8.
  15. Timofte R, De Smet V, Van Gool L. Anchored neighborhood regression for fast example-based super-resolution. IEEE Int Conf on Computer Vision 2013: 1920-1927. DOI: 10.1109/ICCV.2013.241.
  16. Kim J, Lee JK, Lee KM. Deeply-recursive convolutional network for image super-resolution. IEEE Conf on Computer Vision and Pattern Recognition 2016: 1637-1645.
  17. Dong C, Loy CC, He K, Tang X. Learning a deep convolutional network for image super-resolution. In Book: Fleet D, Pajdla T, Schiele B, Tuytelaars T, eds. Computer Vision – ECCV 2014. Cham: Springer, 2014: 184-199. DOI: 10.1007/978-3-319-10593-2_13.
  18. Tai Y, Yang J, Liu X. Image super-resolution via deep recursive residual network. IEEE Conf on Computer Vision and Pattern Recognition (CVPR) 2017: 3147-3155. DOI: 10.1109/CVPR.2017.298.
  19. Han W, Chang S, Liu D, Yu M, Witbrock M, Huang TS. Image super-resolution via dual-state recurrent networks. IEEE/CVF Conf on Computer Vision and Pattern Recognition 2018: 1654-1663. DOI: 10.1109/CVPR.2018.00178.
  20. Wan J, Yin H, Chong AX, Liu ZH. Progressive residual networks for image super-resolution. Appl Intell 2020; 50: 1620-1632. DOI: 10.1007/s10489-019-01548-8.
  21. Ren H, El-Khamy M, Lee J. Image super resolution based on fusing multiple convolution neural networks. IEEE Conf on Computer Vision and Pattern Recognition Workshops (CVPRW) 2017: 1050-1057. DOI: 10.1109/CVPRW.2017.142.
  22. Ulyanov D, Vedaldi A, Lempitsky V. Deep image prior. IEEE Conf on Computer Vision and Pattern Recognition (CVPR) 2018: 9446-9454. DOI: 10.1109/CVPR.2018.00984.
  23. Bulat A, Yang J, Tzimiropoulos G. To learn image super-resolution, use a gan to learn how to do image degradation first. In Book: Ferrari V, Hebert M, Sminchisescu C, Weiss Y, eds. Computer Vision – ECCV 2018. Cham: Springer; 2018: 187-202. DOI: 10.1007/978-3-030-01231-1_12.
  24. Kim J, Lee S. Deep learning of human visual sensitivity in image quality assessment framework. IEEE Conf on Computer Vision and Pattern Recognition (CVPR) 2017: 1969-1977. DOI: 10.1109/CVPR.2017.213.
  25. Huang H, He R, Sun Z, Tan T. Wavelet-SRNet: A wavelet-based CNN for multi-scale face super resolution. IEEE Int Conf on Computer Vision (ICCV) 2017: 1698-1706. DOI: 10.1109/ICCV.2017.187.
  26. Kawulok M, Beneck P, Hrynczenko K, Kostrzewa D, J.Nalepa Deep Learning for Multiple-Image Super-Resolution. IEEE Geosci Remote Sens Lett 2019; 17(6): 1062-1066. DOI: 10.1109/LGRS.2019.2940483.
  27. Greaves A, Winter H. Multi-frame video super-resolution using convolutional neural networks. Source: <http://cs231n.stanford.edu/reports/2016/pdfs/212Report.pdf>.
  28. Morin R, Basarab A, Bidon S, Kouamé D. Motion estimation-based image enhancement in ultrasound imaging. Ultrasonics 2015; 60: 19-26. DOI: 10.1016/j.ultras.2015.02.003.
  29. Rossi M, Frossard P.  Graph-based light field super-resolution. IEEE 19th Int Workshop on Multimedia Signal Processing (MMSP) 2017: 1-6. DOI: 10.1109/MMSP.2017.8122224.
  30. Zhou S, Yuan Y, Su L, Ding X, Wang J. Multiframe super resolution reconstruction method based on light field angular images. Opt Commun 2017; 404: 189-195. DOI: 10.1016/j.optcom.2017.03.019.
  31. Quevedo E, Marrero G, Tobajas F. Approach to super-resolution through the concept of multicamera imaging. In Book: Radhakrishnan S, ed. Recent advances in image and video coding. Ch 5. InTech Open; 2016: 101-123. DOI: 10.5772/65442.
  32. Belov AM, Denisova AY. Spectral and spatial super-resolution method for Earth remote sensing image fusion. Computer Optics 2018; 42(5): 855-863. DOI: 10.18287/2412-6179-2018-42-5-855-863.
  33. Ivankov AJ, Sirota AA. Algorithm to construct the superresolution images in false observations [In Russian]. Proceedings of Voronezh State University. Series: Systems Analysis and Information Technologies 2014; 3: 91-99.
  34. Raschupkin AV. Methods of remote sensing data processing for the improvement of output images quality [In Russian]. Vestnik of Samara University. Aerospace and Mechanical Engineering 2010; 2(22): 124-134.
  35. Maksimov A, Sergeyev V. Optimal fusing of video sequence images. 2020 International Conference on Information Technology and Nanotechnology (ITNT) 2020: 1-4. DOI: 10.1109/ITNT49337.2020.9253202.
  36. Soifer VA, ed. Methods of computer image processing [In Russian]. Moscow: "Fizmatlit" Publisher; 2003. ISBN: 5-9221-0270-2.
  37. Sergeyev VV, Maksimov AI. Comparison of optimum reconstruction filters for discrete and continuous-discrete linear observation models. J Phys Conf Ser 2018; 1096: 012031. DOI:10.1088/1742-6596/1096/1/012031.

© 2009, IPSI RAS
151, Molodogvardeiskaya str., Samara, 443001, Russia; E-mail: journal@computeroptics.ru ; Tel: +7 (846) 242-41-24 (Executive secretary), +7 (846) 332-56-22 (Issuing editor), Fax: +7 (846) 332-56-20