arxiv-2609-04272 SRC-0202 · paperEvaluating Large Language Models for Forced Outage Risk Prediction: Benefits and Comparison to Machine Learning
Christos Petridis, Zoran Obradovic, Mladen Kezunović
Created: 2026-09-02 · Ingested: 2026-09-19
https://arxiv.org/abs/2609.04272(opens in a new tab)This study examines the ability of large language models (LLMs) to predict the risk of weather-related forced outages in the distribution grid in a zero-shot framework, without labeled training data. The problem is formulated as a binary severity classification task across three forecast horizons (3h, 6h, 12h), using six years of outage records and high-resolution weather data for a utility service area in central Texas. Four zero-shot LLMs are benchmarked against two supervised classifiers across two input configurations: one using current weather observations and the other using weather fore
In brief
Zero-shot GPT models reading raw weather features lose to small supervised classifiers on a rare-event grid outage task, and the gap is mostly precision.
The task is binary high/low severity classification of weather-driven forced outages in a central Texas utility service area over 6 years (2018-2023), at 3h, 6h and 12h horizons, with thresholds set to a ~5% positive rate. GPT-4o, GPT-4o-mini, GPT-4.1 and GPT-4.1-mini get an identical prompt with either full-window forecast weather or current observations only; LightGBM and Logistic Regression are trained with Leave-One-Year-Out cross-validation on the same held-out years.
With forecast, LightGBM reaches macro-F1 0.74 at 3h and 0.67 at 12h; the best LLM is 0.61-0.63, a gap of 0.04 to 0.13. GPT-4o-mini collapses into near-constant positive prediction (recall 0.78-0.97, precision 0.05-0.07). Adding forecast weather helps ML consistently (+0.06, +9.8% for LightGBM at 12h) but barely moves GPT-4.1-mini (+0.00 at 3h).
Evidence: measured, same folds for both paradigms, with a weather-versus-outage causality check (Granger F = 2,853 at lag 1h). But 1 utility, 1 model family, no prompt tuning, no threshold calibration, no in-context examples, and the interpretability claim is argued from sample rationales, not tested with operators.
Treat zero-shot LLMs here as a cheap fallback where labels are absent, not a replacement.
Written from the abstract by claude-opus-5 on 2026-09-18, with every figure checked against it. Not a substitute for the paper.
Referenced by
Claims in this catalog that draw on this source, and whether as support or counterpoint.
- supportsOn zero-shot binary risk classification of rare weather-driven power outages from tabular weather features, GPT-4o/4.1-class models trail supervised classifiers trained on the same data by 0.04-0.13 macro-F1 and have much lower precision, and they gain little from being given the full forecast window rather than only current observations, while supervised models gain more from forecast features.
- supportsFor zero-shot risk forecasting from tabular weather features on a rare-event (~5% positive) binary severity task, GPT-4o/4.1 class models trail simple supervised classifiers (logistic regression, LightGBM) trained on the same data on macro-F1 and precision at every 3h/6h/12h horizon, and the weaker models close the gap only by flagging nearly every window as high risk.