A multimodal feature fusion network based on multi-scale data for ultra-short-term global horizontal irradiance forecasting
Author Identifier (ORCID)
Abstract
The inherent intermittency and high-frequency variability of solar irradiance, particularly during rapid cloud advection, present significant stability challenges to high-penetration photovoltaic (PV) grids. Although multimodal forecasting has emerged as a viable mitigation strategy, existing architectures predominantly rely on shallow feature concatenation and binary cloud segmentation, thereby failing to capture the fine-grained optical features of clouds and the complex spatiotemporal coupling between visual and meteorological modalities. To bridge this gap, this paper proposes M3S-Net, a novel MultiModal feature fusion network based on Multi-Scale data for ultra-short-term global horizontal irradiance (GHI) forecasting. First, a multi-scale partial channel selection network leverages partial convolutions to explicitly isolate the boundary features of optically thin clouds, effectively transcending the precision limitations of coarse-grained binary masking. Second, a multi-scale sequence to image analysis network employs fast fourier transform (FFT)-based time–frequency representation to disentangle the complex periodicity of meteorological data across varying time horizons. Crucially, the model incorporates a cross-modal Mamba interaction module featuring a novel dynamic “C-matrix swapping” mechanism. By exchanging state-space parameters between visual and temporal streams, this design conditions the state evolution of one modality on the context of the other, enabling deep structural coupling with linear computational complexity, thus overcoming the limitations of shallow concatenation. Experimental validation on the newly constructed fine-grained GHI prediction dataset (FGPD) demonstrates that M3S-Net achieves a mean absolute error (MAE) of 19.84 W/m2 and a R-squared (R2) of 0.964 for the 10-minute forecast. Compared with the state-of-the-art (SOTA) baselines, M3S-Net reduces the MAE by 6.2%. These results indicate that deep cross-modal interaction and fine-grained visual cloud-feature extraction are potential applications of improved short-horizon irradiance forecasting.
Keywords
cross-modal interaction, ground-based cloud image, Mamba architecture, multimodal feature fusion, ultra-short-term GHI forecasting
Document Type
Journal Article
Date of Publication
11-15-2026
Article Number
115101
ISSN
0038092X
Volume
319
Publication Title
Solar Energy
Publisher
Elsevier
School
School of Business and Law
Funding Information
The research was supported by the National Natural Science Foundation of China under Grant No. 62206085, supported by the Innovation Capacity Improvement Plan Project of Hebei Province under 22567603H, and supported by the Interdisciplinary Postgraduate Training Program of Hebei University of Technology under HEBUT-Y-XKJC- 2022101.
Copyright
subscription content
Recommended Citation
Niu, P., Cai, T., Zhang, S., Gu, J., Zhang, P., Liu, Q., & Li, J. (2026). A multimodal feature fusion network based on multi-scale data for ultra-short-term global horizontal irradiance forecasting. Solar Energy, 319, Article 115101. https://doi.org/10.1016/j.solener.2026.115101