A New and Efficient Pan Sharpening Method Based on Optimized Pixel Coefficients

Year 2024, Volume: 11 Issue: 1, 24 - 40, 28.03.2024

Tuba Çağlıkantar , Melih Can Kılıç

https://doi.org/10.54287/gujsa.1407864

Cited By: 1

Abstract

Pan sharpening aims to create a multispectral, high spatial resolution image by combining the multispectral image (MSI) with a high spatial resolution panchromatic image (PAN). Pan sharpening methods are performed between the MS image, which is the MSI image brought to PAN dimensions with the help of interpolation, and the PAN image. In this study, PAN sharpening is approached as an optimization problem. It is assumed that the optimal solution consists of multiplying the pixels of the MS image by optimized coefficients. It would be costly to optimize all the coefficients in this coefficient matrix one by one. For this reason, these coefficients were tried to be found with 5 different optimizationbased methods. It was also compared with 19 different methods commonly used in the literature. 6 different evaluation criteria were used for this comparison. These comparisons were made on 3 different datasets. It has been observed that the proposed methods are superior to other methods.

Keywords

Pan Sharpening, Image Fusion, Remote Sensing, Optimization

References

• Aiazzi, B., Alparone, L., Baronti, S., & Garzelli, A. (2002). Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis. IEEE Transactions on geoscience and remote sensing, 40(10), 2300-2312. https://doi.org/10.1109/TGRS.2002.803623
• Aiazzi, B., Alparone, L., Baronti, S., Garzelli, A., & Selva, M. (2006). MTF-tailored multiscale fusion of high-resolution MS and Pan imagery. Photogrammetric Engineering & Remote Sensing, 72(5), 591-596. https://doi.org/10.14358/PERS.72.5.591
• Alparone, L., Aiazzi, B., Baronti, S., Garzelli, A., Nencini, F., & Selva, M. (2008). Multispectral and panchromatic data fusion assessment without reference. Photogrammetric Engineering & Remote Sensing, 74(2), 193-200. https://doi.org/10.14358/PERS.74.2.193
• Amro, I., Mateos, J., Vega, M., Molina, R., & Katsaggelos, A. K. (2011). A survey of classical methods and new trends in pansharpening of multispectral images. EURASIP Journal on Advances in Signal Processing, 2011(1), 1-22. https://doi.org/10.1186/1687-6180-2011-79
• Ciotola, M., Poggi, G., & Scarpa, G. (2023). Unsupervised Deep Learning-based Pansharpening with Jointly-Enhanced Spectral and Spatial Fidelity. IEEE Transactions on geoscience and remote sensing. https://doi.org/10.48550/arXiv.2307.14403
• Civicioglu, P., & Besdok, E. (2022). Contrast stretching based pansharpening by using weighted differential evolution algorithm. Expert Systems with Applications, 208, 118144. https://doi.org/10.1016/j.eswa.2022.118144
• Civicioglu, P., & Besdok, E. (2024). Pansharpening of remote sensing images using dominant pixels. Expert Systems with Applications, 242, 122783. https://doi.org/10.1016/j.eswa.2023.122783
• Civicioglu, P., Besdok, E., Gunen, M. A., & Atasever, U. H. (2020). Weighted differential evolution algorithm for numerical function optimization: a comparative study with cuckoo search, artificial bee colony, adaptive differential evolution, and backtracking search optimization algorithms. Neural Computing and Applications, 32, 3923-3937. https://doi.org/10.1007/s00521-018-3822-5
• Feng, Y., Yan, B., Jeon, S., & Yang, X. (2024). A hyperspectral pansharpening method using retrain transformer network for remote sensing images in UAV communications system. Wireless Networks, 1-14. https://doi.org/10.1007/s11276-023-03611-2
• Garzelli, A. (2014). Pansharpening of multispectral images based on nonlocal parameter optimization. IEEE Transactions on geoscience and remote sensing, 53(4), 2096-2107. https://doi.org/10.1109/TGRS.2014.2354471
• Gezgin. (2013). (Accessed:01/11/2023) https://gezgin.gov.tr/geoportal/app/main?execution=e3s1
• Ghassemian, H. (2016). A review of remote sensing image fusion methods. Information Fusion, 32, 75-89. https://doi.org/10.1016/j.inffus.2016.03.003
• Günen, M. A. (2021). Weighted differential evolution algorithm based pansharpening. International Journal of Remote Sensing, 42(22), 8468-8491. https://doi.org/10.1080/01431161.2021.1976874
• Kallel, A. (2014). MTF-adjusted pansharpening approach based on coupled multiresolution decompositions. IEEE Transactions on geoscience and remote sensing, 53(6), 3124-3145. https://doi.org/10.1109/TGRS.2014.2369056
• Kurban, T. (2022). Region based multi-spectral fusion method for remote sensing images using differential search algorithm and IHS transform. Expert Systems with Applications, 189, 116135. https://doi.org/10.1016/j.eswa.2021.116135
• Lolli, S., Alparone, L., Garzelli, A., & Vivone, G. (2017). Haze correction for contrast-based multispectral pansharpening. IEEE Geoscience and Remote Sensing Letters, 14(12), 2255-2259. https://doi.org/10.1109/LGRS.2017.2761021
• Masi, G., Cozzolino, D., Verdoliva, L., & Scarpa, G. (2016). Pansharpening by convolutional neural networks. Remote Sensing, 8(7), 594. https://doi.org/10.3390/rs8070594
• Meng, X., Shen, H., Li, H., Zhang, L., & Fu, R. (2019). Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis: Practical discussion and challenges. Information Fusion, 46, 102-113. https://doi.org/10.1016/j.inffus.2018.05.006
• Meng, X., Xiong, Y., Shao, F., Shen, H., Sun, W., Yang, G., Yuan, Q., Fu, R., & Zhang, H. (2020). A large-scale benchmark data set for evaluating pansharpening performance: Overview and implementation. IEEE Geoscience and Remote Sensing Magazine, 9(1), 18-52. https://doi.org/10.1109/MGRS.2020.2976696
• Open Remote Sensing. (2015). (Accessed:01/11/2023) https://openremotesensing.net/knowledgebase/a-critical-comparison-among-pansharpening-algorithms/
• Otazu, X., González-Audícana, M., Fors, O., & Núñez, J. (2005). Introduction of sensor spectral response into image fusion methods. Application to wavelet-based methods. IEEE Transactions on geoscience and remote sensing, 43(10), 2376-2385. https://doi.org/10.1109/TGRS.2005.856106
• Palsson, F., Sveinsson, J. R., & Ulfarsson, M. O. (2013). A new pansharpening algorithm based on total variation. IEEE Geoscience and Remote Sensing Letters, 11(1), 318-322. https://doi.org/10.1109/LGRS.2013.2257669
• Restaino, R., Vivone, G., Dalla Mura, M., & Chanussot, J. (2016). Fusion of multispectral and panchromatic images based on morphological operators. IEEE Transactions on Image Processing, 25(6), 2882-2895. https://doi.org/10.1109/tip.2016.2556944
• Scarpa, G., Vitale, S., & Cozzolino, D. (2018). Target-adaptive CNN-based pansharpening. IEEE Transactions on geoscience and remote sensing, 56(9), 5443-5457. https://doi.org/10.1109/TGRS.2018.2817393
• Shahdoosti, H. R., & Ghassemian, H. (2014). Fusion of MS and PAN images preserving spectral quality. IEEE Geoscience and Remote Sensing Letters, 12(3), 611-615. https://doi.org/10.1109/LGRS.2014.2353135
• Tu, T., Su, S., Shyn, H., & Huang, P. (2001). A new look at IHS-like image fusion methods, Information Fusion. https://doi.org/10.1016/S1566-2535(01)00036-7
• Vivone, G. (2019). Robust band-dependent spatial-detail approaches for panchromatic sharpening. IEEE Transactions on geoscience and remote sensing, 57(9), 6421-6433. https://doi.org/10.1109/TGRS.2019.2906073
• Vivone, G., Alparone, L., Chanussot, J., Dalla Mura, M., Garzelli, A., Licciardi, G. A., Restaino, R., & Wald, L. (2014a). A critical comparison among pansharpening algorithms. IEEE Transactions on geoscience and remote sensing, 53(5), 2565-2586. https://doi.org/10.1109/TGRS.2014.2361734
• Vivone, G., Dalla Mura, M., Garzelli, A., Restaino, R., Scarpa, G., Ulfarsson, M. O., Alparone, L., & Chanussot, J. (2020). A new benchmark based on recent advances in multispectral pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods. IEEE Geoscience and Remote Sensing Magazine, 9(1), 53-81. https://doi.org/10.1109/MGRS.2020.3019315
• Vivone, G., Restaino, R., & Chanussot, J. (2017). A regression-based high-pass modulation pansharpening approach. IEEE Transactions on geoscience and remote sensing, 56(2), 984-996. https://doi.org/10.1109/TGRS.2017.2757508
• Vivone, G., Restaino, R., & Chanussot, J. (2018). Full scale regression-based injection coefficients for panchromatic sharpening. IEEE Transactions on Image Processing, 27(7), 3418-3431. https://doi.org/10.1109/TIP.2018.2819501
• Vivone, G., Restaino, R., Dalla Mura, M., Licciardi, G., & Chanussot, J. (2013). Contrast and error-based fusion schemes for multispectral image pansharpening. IEEE Geoscience and Remote Sensing Letters, 11(5), 930-934. https://doi.org/10.1109/LGRS.2013.2281996
• Vivone, G., Simões, M., Dalla Mura, M., Restaino, R., Bioucas-Dias, J. M., Licciardi, G. A., & Chanussot, J. (2014b). Pansharpening based on semiblind deconvolution. IEEE Transactions on geoscience and remote sensing, 53(4), 1997-2010. https://doi.org/10.1109/TGRS.2014.2351754
• Wald, L., Ranchin, T., & Mangolini, M. (1997). Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images. Photogrammetric engineering and remote sensing, 63(6), 691-699.
• Yilmaz, C. S., Yilmaz, V., & Gungor, O. (2022). A theoretical and practical survey of image fusion methods for multispectral pansharpening. Information Fusion, 79, 1-43. https://doi.org/10.1016/j.inffus.2021.10.001