When people talk about weather forecasts, the most common question is simple: is it accurate? If the only standard is avoiding missed events, the answer is surprisingly easy. Forecast rain every single day. You will never miss a rainy day. Your hit rate will reach 100%. It looks impressive. It means nothing.
In forecast verification, one of the most basic metrics is POD, the Probability of Detection. It is calculated as the number of correctly forecast events divided by the number of events that actually occurred. Suppose there are 100 rainy days in a year. As long as all 100 are forecast, the POD is 100%. If you predict rain every day, this condition is automatically satisfied. Even someone with no judgment at all can achieve a perfect score on paper.
But the problem appears immediately. On the remaining 265 days, it does not rain. Yet all of them are forecast as rain. These are false alarms. That is where FAR, the False Alarm Ratio, becomes relevant. FAR is calculated as the number of false alarms divided by the total number of forecast events. If rain is forecast every day, then out of 365 forecasts, only 100 are correct and 265 are false alarms. The FAR is about 73%. In other words, more than seven out of ten warnings are unnecessary. Trust quickly erodes under such conditions.
This is why POD alone is meaningless. A forecast can inflate its detection rate through extreme strategies while simultaneously amplifying false alarms. On the other hand, if rain is never forecast, the FAR is zero. Yet every rainy day is missed. The POD falls to zero. That is equally useless. The real challenge lies in finding a reasonable position between these extremes.
Meteorologists often use CSI, the Critical Success Index, to assess hits, false alarms and misses together. CSI is calculated as the number of hits divided by the sum of hits, false alarms and misses. If a forecast is too aggressive or too conservative, CSI will remain low. Only with balanced judgment does the index improve. This metric forces forecasters to take responsibility for overall performance rather than hide behind a single flattering number.
At its core, this is a question of risk management. If the cost of missing an event is extremely high, such as heavy rainfall triggering landslides, a higher false alarm rate may be acceptable. If the cost of false alarms is substantial, such as unnecessary school closures or economic disruption, then false alarms must be tightly controlled. Forecasting is never a simple competition about being right or wrong. It is a trade-off between costs and risks.
What is described above concerns basic categorical forecast verification. Modern weather forecasting increasingly uses probabilistic formats, such as predicting a 30% or 70% chance of rain. Verifying probabilistic forecasts involves deeper concepts such as reliability, resolution and the Brier Score. These are far more complex than POD or CSI. Determining whether a probabilistic forecast is both reliable and capable of distinguishing different outcomes is another layer of evaluation. That discussion will have to wait.
The idea of being 100% accurate is often little more than a definitional trick. Choose a metric that favours your strategy and impressive numbers will follow. Responsible forecasting requires multiple metrics and clear explanations of trade-offs. Numbers do not speak for themselves. People decide which numbers to emphasise.
What applies to weather forecasts applies equally to other forms of prediction. If we chase surface accuracy without confronting costs and uncertainty, any prediction can be made to look perfect. The question is not whether perfection is achievable. The question is whether we are willing to measure meaning honestly.
