A Real-Time Learning-Based Super-Resolution System on FPGA
Abstract
This paper proposes a real-time super-resolution (SR) system. The proposed system performs a fast SR algorithm that generates a high-resolution image from a low-resolution image using direct regression functions with an up-scaling factor of 2. This algorithm contained two processes: feature learning and SR image prediction. The feature learning stage is performed offline, in which several regression functions were trained. The SR image prediction stage is implemented on the proposed system to generate high-resolution image patches. The system implemented on a Xilinx Virtex 7 field-programmable gate array achieves output resolution of (UHD) at 85 fps and 700Mpixels/s throughput. Structure similarity (SSIM) is measured for image quality. Experimental results show that the proposed system provides high image quality for real-time applications. And the proposed system possesses high scalability for resolution.
References
- 1. , Boosting performance and speed of single-image super-resolution based on partitioned linear regression, IEEE International Conference on Image Processing (IEEE, 2016), pp. 1419–1423. Crossref, Google Scholar
- 2. , Image super-resolution via sparse representation, IEEE Transactions on Image Processing 19(11) (2010) 2861–2873. Crossref, ISI, Google Scholar
- 3. W. S. Lai, J. B. Huang, N. Ahuja et al., Deep Laplacian pyramid networks for fast and accurate super-resolution (2017). Google Scholar
- 4. , Real-time image super resolution using an FPGA, International Conference on Field Programmable Logic and Applications (IEEE, 2008), pp. 89–94. Crossref, Google Scholar
- 5. G. Freedman and R. Fattal, Image and video upscaling from local self-examples (ACM, 2011). Google Scholar
- 6. , FPGA implementation of single-image super-resolution based on frame-Bufferless box filtering, Journal of Signal Processing 17(4) (2013) 111–114. Crossref, Google Scholar
- 7. , Hardware implementation of real-time multiple frame super-resolution, IFIP/IEEE International Conference on Very Large Scale Integration (IEEE, 2015), pp. 219–224. Crossref, Google Scholar
- 8. , An embedded hardware architecture for real-time super-resolution in infrared cameras, Digital System Design (IEEE, 2016), pp. 184–191. Crossref, Google Scholar
- 9. , A real-time FHD learning-based super-resolution system without a frame buffer, IEEE Transactions on Circuits and Systems II Express Briefs (
2017 ), PP(99):1–1. Google Scholar - 10. , 2X super-resolution hardware using edge-orientation-based linear mapping for real-time 4K UHD 60 fps video applications, IEEE Transactions on Circuits and Systems II: Express Briefs (
2018 ). Google Scholar - 11. , Fast direct super-resolution by simple functions, IEEE International Conference on Computer Vision (IEEE, 2014), pp. 561–568. Google Scholar
- 12. , Error detecting and error correcting codes, Bell System Technical Journal 29(2) (1950) 147–160. Crossref, ISI, Google Scholar
- 13. , Iterative linear interpolation based on fuzzy gradient model for low-cost VLSI implementation, IEEE Transactions on Very Large Scale Integration Systems 22(7) (2014) 1526–1538. Crossref, ISI, Google Scholar
- 14. , High-performance low-area video up-scaling architecture for 4-K UHD Video, IEEE Transactions on Circuits and Systems II Express Briefs 64(4) (2017) 437–441. Crossref, ISI, Google Scholar
- 15. , FPGA implementation of a real-time super-resolution system using a convolutional neural network, International Conference on Field-Programmable Technology (IEEE, 2017), pp. 249–252. Google Scholar
- 16. , Image quality assessment: From error visibility to structural similarity, IEEE Transactions on Image Processing 13(4) (2004) 600. Crossref, ISI, Google Scholar
- 17. , Winograd-based real-time super-resolution system on FPGA, 2019 International Conference on Field-Programmable Technology (ICFPT) (
Tianjin, China ,2019 ), pp. 423–426. https://doi.org/10.1109/ICFPT47387.2019.00083 Google Scholar - 18. , An energy-efficient FPGA-Based deconvolutional neural networks accelerator for single image super-resolution, in IEEE Transactions on Circuits and Systems for Video Technology 30(1) (2020) 281–295. https://doi.org/10.1109/TCSVT.2018.2888898 Crossref, ISI, Google Scholar


