Haobo LI

Publications

1. 

   PIPE: Physics-Informed Position Encoding for Alignment of Satellite Images and Time Series

   Haobo Li, Eunseo Jung, Zixin Chen, and 4 more authors

   NeurIPS 2025

   Abs Bib PDF

   Multimodal time series forecasting is foundational in various fields, such as utilizing satellite imagery and numerical data for predicting typhoons in climate science. However, existing multimodal approaches primarily focus on utilizing text data to help time series forecasting, leaving the visual data in existing time series datasets untouched. Furthermore, it is challenging for models to effectively capture the physical information embedded in visual data, such as satellite imagery’s temporal and geospatial context, which extends beyond images themselves. To address this gap, we propose physics-informed positional encoding (PIPE), a lightweight method that embeds physical information into vision language models (VLMs). PIPE introduces two key innovations: (1) a physics-informed positional indexing scheme for mapping physics to positional IDs, and (2) a variant-frequency positional encoding mechanism for encoding frequency information of physical variables and sequential order of tokens within the embedding space. By preserving both the physical information and sequential order information, PIPE significantly improves multimodal alignment and forecasting accuracy. Through the experiments on the most representative and the largest open-sourced satellite image dataset, PIPE achieves state-of-the-art performance in both deep learning forecasting and climate domain methods, demonstrating superiority across benchmarks, including a 12% improvement in typhoon intensity forecasting over priorworks. Our code is provided in the supplementary material.

   @article{li2025pipe,
     author = {Li, Haobo and Jung, Eunseo and Chen, Zixin and Wang, Zhaowei and Yueya, Wang and Qu, Huamin and Lau, Alexis Kai Hon},
     journal = {NeurIPS},
     title = {PIPE: Physics-Informed Position Encoding for Alignment of Satellite Images and Time Series},
     year = {2025},
   }

2. 

   CLLMate: A Multimodal Benchmark for Weather and Climate Events Forecasting

   Haobo Li, Zhaowei Wang, Jiachen Wang, and 3 more authors

   EMNLP Main Conference 2025

   Abs arXiv Bib PDF

   Forecasting weather and climate events is crucial for making appropriate measures to mitigate environmental hazards and minimize associated losses. Previous research on environmental forecasting focuses on predicting numerical meteorological variables related to closed-set events rather than forecasting open-set events directly, which limits the comprehensiveness of event forecasting. We propose Weather and Climate Event Forecasting (WCEF), a new task that leverages meteorological raster data and textual event data to predict potential weather and climate events. However, due to difficulties in aligning multimodal data and the lack of sufficient supervised datasets, this task is challenging to accomplish. Therefore, we first propose a framework to align historical meteorological data with past weather and climate events using the large language model (LLM). In this framework, we construct a knowledge graph by using LLM to extract information about weather and climate events from a corpus of over 41k highly environment-focused news articles. Subsequently, we mapped these events with meteorological raster data, creating a supervised dataset, which is the largest and most novel for LLM tuning on the WCEF task. Finally, we introduced our aligned models, CLLMate (LLM for climate), a multimodal LLM to forecast weather and climate events using meteorological raster data. In evaluating CLLMate, we conducted extensive experiments. The results indicate that CLLMate surpasses both the baselines and other multimodal LLMs, showcasing the potential of utilizing LLM to align weather and climate events with meteorological data and highlighting the promising future for research on the WCEF task. As a representative hub in the “My Climate Risk” lighthouse activity initiated by the World Climate Research Programme (WCRP), we contribute CLLMate as a component of our regional solution to help forecast environmental risks and mitigate their loss.

   @article{li2025cllmate,
     title = {CLLMate: A Multimodal Benchmark for Weather and Climate Events Forecasting},
     author = {Li, Haobo and Wang, Zhaowei and Wang, Jiachen and Wang, Yueya and Lau, Alexis Kai Hon and Qu, Huamin},
     journal = {EMNLP Main Conference},
     year = {2025},
     doi = {https://doi.org/10.48550/arXiv.2409.19058},
   }

3. 

   Save It for the "Hot" Day: An LLM-Empowered Visual Analytics System for Heat Risk Management

   Haobo Li, Wong Kam-Kwai, Yan Luo, and 6 more authors

   IEEE Transactions on Visualization and Computer Graphics 2025

   Abs arXiv Bib PDF

   The escalating frequency and intensity of heat-related climate events, particularly heatwaves, emphasize the pressing need for advanced heat risk management strategies. Current approaches, primarily relying on numerical models, face challenges in spatial-temporal resolution and in capturing the dynamic interplay of environmental, social, and behavioral factors affecting heat risks. This has led to difficulties in translating risk assessments into effective mitigation actions. Recognizing these problems, we introduce a novel approach leveraging the burgeoning capabilities of Large Language Models (LLMs) to extract rich and contextual insights from news reports. We hence propose an LLM-empowered visual analytics system, Havior, that integrates the precise, data-driven insights of numerical models with nuanced news report information. This hybrid approach enables a more comprehensive assessment of heat risks and better identification, assessment, and mitigation of heat-related threats. The system incorporates novel visualization designs, such as “thermoglyph” and news glyph, enhancing intuitive understanding and analysis of heat risks. The integration of LLM-based techniques also enables advanced information retrieval and semantic knowledge extraction that can be guided by experts’ analytics needs. We collaborated with six domain experts to conduct a case study on the 2022 China Heatwave and an expert survey & interview, demonstrating the usefulness of our system in providing in-depth and actionable insights for heat risk management.

   @article{li2025save,
     title = {Save It for the "Hot" Day: An LLM-Empowered Visual Analytics System for Heat Risk Management},
     author = {Li, Haobo and Kam-Kwai, Wong and Luo, Yan and Chen, Juntong and Liu, Chengzhong and Zhang, Yaxuan and Lau, Alexis Kai Hon and Qu, Huamin and Liu, Dongyu},
     journal = {IEEE Transactions on Visualization and Computer Graphics},
     year = {2025},
     doi = {10.1109/TVCG.2025.3586689},
     url = {https://ieeexplore.ieee.org/document/11072384},
   }

4. 

   CoGrader: Transforming Instructors’ Assessment of Project Reports through Collaborative LLM Integration

   Zixin Chen, Jiachen Wang, Haobo Li, and 3 more authors

   ACM UIST 2025

   Abs Bib PDF

   Grading project reports are increasingly significant in today’s educational landscape, where they serve as key assessments of students’ comprehensive problem-solving abilities. However, it remains challenging due to the multifaceted evaluation criteria involved, such as creativity and peer-comparative achievement. Meanwhile, instructors often struggle to maintain fairness throughout the time-consuming grading process. Recent advances in AI, particularly large language models, have demonstrated potential for automating simpler grading tasks, such as assessing quizzes or basic writing quality. However, these tools often fall short when it comes to complex metrics, like design innovation and the practical application of knowledge, that require an instructor’s educational insights into the class situation. To address this challenge, we conducted a formative study with six instructors and developed CoGrader, which introduces a novel grading workflow combining human-LLM collaborative metrics design, benchmarking, and AI-assisted feedback. CoGrader was found effective in improving grading efficiency and consistency while providing reliable peer-comparative feedback to students. We also discuss design insights and ethical considerations for the development of human-AI collaborative grading systems.

   @article{chen2025cograder,
     title = {CoGrader: Transforming Instructors' Assessment of Project Reports through Collaborative LLM Integration},
     author = {Chen, Zixin and Wang, Jiachen and Li, Haobo and Shi, Chuhan and Zhang, Rong and Qu, Huamin},
     journal = {ACM UIST},
     year = {2025},
   }

5. 

   Exploring Spatial Hybrid User Interface for Visual Sensemaking

   Wai Tong, Haobo Li, Meng Xia, and 4 more authors

   IEEE Transactions on Visualization and Computer Graphics 2025

   Abs Bib PDF

   We built a spatial hybrid system that combines a personal computer (PC) and virtual reality (VR) for visual sensemaking, addressing limitations in both environments. Although VR offers immense potential for interactive data visualization (e.g., large display space and spatial navigation), it can also present challenges such as imprecise interactions and user fatigue. At the same time, a PC offers precise and familiar interactions but has limited display space and interaction modality. Therefore, we iteratively designed a spatial hybrid system (PC+VR) to complement these two environments by enabling seamless switching between PC and VR environments. To evaluate the system’s effectiveness and user experience, we compared it to using a single computing environment (i.e., PC-only and VR-only). Our study results (N=18) showed that spatial PC+VR could combine the benefits of both devices to outperform user preference for VR-only without a negative impact on performance from device switching overhead. Finally, we discussed future design implications.

   @article{tong2025exploring,
     author = {Tong, Wai and Li, Haobo and Xia, Meng and Kam-Kwai, Wong and Pong, Ting-Chuen and Qu, Huamin and Yang, Yalong},
     journal = {IEEE Transactions on Visualization and Computer Graphics},
     title = {Exploring Spatial Hybrid User Interface for Visual Sensemaking},
     year = {2025},
     pages = {1-16},
     keywords = {Data visualization;Visualization;Three-dimensional displays;Switches;Navigation;Keyboards;Hands;User experience;Rendering (computer graphics);Visual analytics;Hybrid user interface;data visualization;node-link diagram;visual sensemaking;document analysis},
     doi = {10.1109/TVCG.2025.3538771},
   }

6. 

   Assessing the Impact of Cumulus Convection and Turbulence Parameterizations on Typhoon Precipitation Forecast

   Yueya Wang, Haobo Li, Xiaoming Shi, and 1 more author

   Geophysical Research Letters 2024

   Abs Bib PDF

   Improving typhoon precipitation forecast with convection-permitting models remains challenging. This study investigates the influence of cumulus parameterizations and turbulence models, including the Reconstruction and Nonlinear Anisotropy (RNA) turbulence scheme, on precipitation prediction in multiple typhoon cases. Incorporating the cumulus and RNA schemes increases domain-averaged precipitation, improves recall scores, and lowers relative error across various precipitation thresholds, which is substantial in three out of four studied typhoon cases. Applying appropriate cumulus parameterization schemes alone also contributes to enhancing heavy precipitation forecasts. In Typhoon Hato, the RNA and Grell-3 schemes demonstrated a doubled recall rate for extreme rainfall compared to simulations without any cumulus scheme. The improved forecasting ability is attributed to the RNA’s capacity to model dissipation and backscatter. The RNA scheme can dynamically reinforce typhoon circulation with upgradient momentum transport in the lower troposphere and enhance the buoyancy by favorable heat flux distribution, which is conducive to developing heavy precipitation.

   @article{wang2024assessing,
     title = {Assessing the Impact of Cumulus Convection and Turbulence Parameterizations on Typhoon Precipitation Forecast},
     author = {Wang, Yueya and Li, Haobo and Shi, Xiaoming and Fung, Jimmy},
     journal = {Geophysical Research Letters},
     year = {2024},
     url = {https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2024GL112075},
   }