Tri-Modal Deep Learning Forecasting Framework to Enhance Performance of Tropical Cyclone Forecasting
Main Article Content
Abstract
Even with these advances, the complex nature of climatic processes in the atmosphere, sparse observation coverage, diverse data sources, and the dynamic nature of tropical cyclones (TCs) remain a challenge to predicting TC tracks and intensity. The existing deep learning approaches primarily rely on unimodal or bimodal observations, and frequently assume full observation or fully synchronized input, which are not always applicable and robust to observation deficient and data-scarce settings. The present work suggests a novel approach of taking a multi-modal generative deep learning approach to jointly predict the track and intensity of TCs using satellite imagery, atmospheric meteorological variables and historical cyclone track sequences. The results are hypothesized to be learnt by using an Improved Vision Transformer for spatial and structural information and historical cyclone trajectories and reanalysis variables for temporal and atmospheric information. Generative models will be discussed for reconstructing missing modalities and information when it is corrupted to enable forecasting when information is missing. The relationships between the three modalities will then be learned with a cross-attention-based fusion mechanism and physics-informed constraints will be added to enhance the physical consistency of forecasts. The proposed framework will also at the same time estimate the future parameters of cyclones like central pressure and maximum sustained wind. Its usefulness will be evaluated against well-available tropical cyclone track and intensity data from various basins by the errors in track and intensity, and the ability to reconstruct the cyclones, as well as by comparative experiments with available deep learning and multimodal track and intensity forecasting models. The aim is to produce a reliable and generally applicable forecasting framework that would maintain good forecasting skills when meteorological data is limited or not available.