Document Type
Journal Article
Publication Title
Sensors
ISSN
14248220
Volume
20
Issue
22
First Page
1
Last Page
17
Publisher
MDPI
School
School of Science
RAS ID
32432
Funders
National Science Foundation of China China Postdoctoral Science Foundation National Science Foundation of Shaan ARC Discovery Grant NVIDIA Corporation
Abstract
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. Convolutional neural networks have recently been used for multi-focus image fusion. However, some existing methods have resorted to adding Gaussian blur to focused images, to simulate defocus, thereby generating data (with ground-truth) for supervised learning. Moreover, they classify pixels as ‘focused’ or ‘defocused’, and use the classified results to construct the fusion weight maps. This then necessitates a series of post-processing steps. In this paper, we present an end-to-end learning approach for directly predicting the fully focused output image from multi-focus input image pairs. The suggested approach uses a CNN architecture trained to perform fusion, without the need for ground truth fused images. The CNN exploits the image structural similarity (SSIM) to calculate the loss, a metric that is widely accepted for fused image quality evaluation. What is more, we also use the standard deviation of a local window of the image to automatically estimate the importance of the source images in the final fused image when designing the loss function. Our network can accept images of variable sizes and hence, we are able to utilize real benchmark datasets, instead of simulated ones, to train our network. The model is a feed-forward, fully convolutional neural network that can process images of variable sizes during test time. Extensive evaluation on benchmark datasets show that our method outperforms, or is comparable with, existing state-of-the-art techniques on both objective and subjective benchmarks.
DOI
10.3390/s20226647
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Comments
Yan, X., Gilani, S. Z., Qin, H., & Mian, A. (2020). Structural similarity loss for learning to fuse multi-focus images. Sensors, 20(22), article 6647. https://doi.org/10.3390/s20226647