Forecasting and Philosophy

I noted yesterday that:

“Weather forecasting (and climate prediction) is not just about computer power. Deep philosophical ideas also come into play.”

I fear I may have under-delivered on the philosophy.

I intended to suggest that all forecasts, such as of weather, are necessarily and systematically inaccurate.

To recap, my main point yesterday was that running an inaccurate forecasting model numerous times doesn’t solve all the inherent problems:

“All ensemble forecasters know is that a certain proportion of computer model runs produce a given outcome. This might help identify possible weather events, but doesn’t tell you real-world probabilities. If there is some factor that the computer model doesn’t take account of then running the computer model 50 times is in effect to make the same mistake 50 times.”

Let me elaborate.

We can dismiss the normal explanation for forecasting difficulties.  Forecasters normally plead “chaos”.  Perfect forecasts are impossible, they say, because the flap of a butterfly’s wings can cause a hurricane.  Small changes in initial conditions can have dramatic consequences.

I don’t accept this excuse for minute.

It may well be the case that computer models suffer badly from the chaos problem.  In fact, the ensemble modelling approach relies on it.  I suspect the real world is much less susceptible.  Besides, given enough computer power I could model the butterfly down to the last molecule and predict its wing-flapping in precise detail.

No, the real-world is determined. That is, there is only one possible outcome.  Given enough information and processing power you could, in principle, predict the future with complete accuracy.

Of course, there are insurmountable practical problems that prevent perfect forecasting:

• The most fundamental difficulty is that no computer can exceed the computing capabilities of the universe itself.  Although the future is written, it is in principle impossible to read it.
• You might try to get round the computing capacity problem by taking part of the universe as a closed system and building a huge computer to model what is going on in that relatively small part.  The difficulty then is that the entire universe is interconnected.  Every part of it is open, not closed.  If the small part you were modelling were the Earth, say, then you’d have to also model all celestial events, not just those which might have a physical effect, but all those which might be detectable by humans and therefore able to affect thought-processes and decision-making.  And, since our telescopes can see galaxies billions of light-years away, there’s a lot to include in your model.  That’s not all, though.  You’d also need to model every cosmic ray that might disrupt a molecule, most dramatically of germ-line DNA – though a change to any molecule is of consequence – and even every photon that might warm a specific electron, contribute to photosynthesis or allow a creature to see just that bit better when hunting…
• Then there are problems of what George Soros terms reflexivity.  That is, people’s behaviour is modified by knowing the predicted future.  They might act to deliberately avoid the modelled outcome, for example by deflecting an asteroid away from its path towards the Earth, which we might term strong reflexivity.  Or they might change their behaviour in a way that unintentionally affects the future, for example by cancelling travel plans in light of a weather forecast – weak reflexivity.  With enough computer power, some such problems could conceivably be overcome.  One might predict the response to an inbound asteroid, for example.  But it’s not immediately apparent how a model would handle the infinitely recursive nature of the general problem.

In practice, of course, these would be nice problems to have, because computer simulations of the weather system are grossly simplified.   They must therefore be systematically biased in their forecasting of any phenomena that rely on the complexity absent from the models.  As I noted yesterday, all runs in an ensemble forecast will suffer from any underlying bias in the model.

Two categories of simplification are problematic:

• Models divide the real-world into chunks, for example layers of the atmosphere (or of the ocean).
• And models necessarily represent closed systems – since the only truly closed system is the universe as a whole.  Anything not included in the model can affect the forecast.  For example, volcanic eruptions will invalidate any sufficiently long-term (and on occasion short-term) weather forecast.  Worse, weather models may be atmosphere only or include only a crude simplification of the oceans.  That is, they may represent the oceans in insufficient detail, and furthermore fail to include the effect of the forecast on the oceans, which in turn affects the forecast later on.

The good news, of course, is that it is possible to improve our weather forecasting almost indefinitely.

Perhaps those presenting weather forecasts should reflect on the fact that, as computer models improve, ensemble forecast ranges will narrow.  The 5-day forecast today is as good as the 3-day forecast was a decade or two ago. The probability of specific real-world conditions will not have changed.  That has always been and always will be precisely one: certainty.

It makes no sense to say “the probability of snow over Easter is x%”, when x depends merely on how big a computer you are using.

No, forecasters need to say instead that “x% of our forecasts predict snow over Easter”, which is not the same thing at all.