Philosophy of Science: Mathematical Modelling, Assumptions and Causal Realism

Q: How can mathematical modelling succeed when it inevitably involves false assumptions?

We happen to know from the history of science that models with false assumptions can be used to make systematically correct predictions with well-understood conditions of application. Kepler's laws of planetary motion are a canonical example [Colyvan & Ginzburg [2003]]; a more dramatic illustration is provided by the very considerable improvements in the power of metereological and climate-modelling since the 1950s. ¹ The very general explanation for this very general fact is that the dynamics---the progression of state changes---of even complex systems seem to causally depend to a large degree on changes in only a small number of mathematically isolable variables which we can track with mathematical equations. This convenience given us by nature does not, however, entail scientific convenience; instead, it has the consequence that truly successful model-building in higher-level sciences will invariably require the identification of the critical causal variables---that is, the right theory or understanding. This methodological truth sits very uncomfortably with the fact that, precisely because of the large number of variables in play in the large-system modelling in the higher-level sciences, there seems to be very little hope for decisive empirical disconfirmation or falsification of models (embodying theories) which are at least vaguely appropriate for the target system. ²

My thesis in this essay is that these considerations ought to lead one to the view that assumptions do, in fact, really matter: more specifically, that the laissez-faire instrumentalist attitude towards assumptions within scientific theories (which has historically held most sway within the discipline of economics) is deeply misguided. I suggest that the history of science (including recent science, and recent developments within ecology) gives strong weight to the view that models which do not embody the causal structure of their target system either don't have predictive success at all, or eventually badly misfire, on account of being attuned only to some subset of the states of the system, rather than locking into the causal structure which determines the state changes of the system. I submit that the key source of instrumentalist error is the failure to recognise that there are different kinds of modelling assumptions, with different scientific significance, and that models which are not `realistic' in terms of their modelling of causal structure cannot succeed (at least in the long-run). I argue that ecology is an excellent case study for this thesis, by attempting to show that causal realism and the absence of highly unrealistic domain assumptions [Musgrave [1981]] is a feature of successful and commonly used models in ecology.

In his paper “Models and Fictions", the philosopher of science Peter Godfrey-Smith makes much of the fact that the scientific enterprise depends on mathematical fiction in the course of asking how it is that fictions in general can tell us about the world [Godfrey-Smith [2009]]. Left unexplored by Godfrey-Smith in the short paper is the fact that in the realm of science we call “fundamental physics", the history and current practice of the science gives us reason to think that our fundamental reality may be perfectly describable using mathematics.³ Godfrey-Smith does himself observe that one of the viable explanations for the problem

¹ More dramatic since, although the amount of chemical and atmospheric `variables' explicitly encoded in climate models has increased over the last few decades and climate models are complicated enough to require state-of-the-art computing power to run, even our best climate models are still (perforce) attempts only to capture the `core' causal structure of the dynamics of

the chaotic target systems [Lucarini [2013]].

²There always exist escape clauses in the form of phrases like “exogenous shocks" and “disturbing factors". To clarify, I do not mean to convey a nihilistic, defeatist or relativist attitude towards the possibility of rational rejection of theories within the higher-level sciences (leading to genuine scientific progress); it will become clear that I believe, in fact, that models which have patently causally unreal assumptions (models which clearly embody the wrong theory) should be dismissed out of hand regardless of any apparent early empirical confirmation. Unfortunately, in the social sciences, there really does seem to be very little hope at all that such norms could be acceded to by a majority, because of the role of ideology and other social

complications.

³The theories of “fundamental physics" really do seem to be profoundly non- fictional, not embodying any assumptions known to be false in the way that higher-level sciences must. The General Theory of Relativity does not model spacetime as if it were a four-dimensional `pseudo-Riemannian' manifold; instead, we have no other grasp on what spacetime is. Similarly, Quantum Electrodynamics (the relativistic quantum field theory of electrodynamics, which Feynmann called the “jewel of

of how fictional mathematical models can tell us about the world is that many of the features of “reality" we assume away to design a good mathematical model are just ornaments for the fundamental universal mathematical patterns inherent in nature---such that a good mathematical model of a system, making the right kind of assumptions, simply cuts through the frippery. ⁴ Whatever the merits of Platonism, I claim that we have very good reason to believe that a very humanly useful approximation of the `causal architecture' of even highly complex or chaotic natural systems is isolable by means of mathematical equations encoding the dominant (numerically abstractable) dependencies of the system. I claim this in view of our knowledge that the necessary presence of the ceteris paribus clause and the “idealisation" in the canonical models of science as we move beyond fundamental physics into non-fundamental physics (and beyond non-fundamental physics into the rest) are not handicaps to very impressive predictions. As I noted in the introduction, it is not just the familiar tales of early Enlightenment physics that bear this out; one of the less appreciated major scientific success stories of the second half of the 20th Century is the constant leaps and bounds we have made in computer-modelling of the earth's weather and climate since the process started with the primitive computers of the 1950s. A 2015 paper in Nature entitled “The Quiet Revolution of Numerical Weather Prediction" gives us some of the key facts testifying to the magnitude this achievement: “Forecast skill in the range from 3 to 10 days ahead has been increasing by about one day per decade" whilst (as an example of some of the progress in longer-term forecasting that has been made) “tropical sea surface temperature variability following the El Nino/Southern Oscillation phenomenon can be predicted 3-4 months ahead" [Bauer et al. 2015, 47].

Unlike Godfrey-Smith (assuming I interpret him correctly), I reject the position that there is any kind of philosophically interesting analogy to be had between novels or films or other obvious fictions, and successful mathematical models of complex systems used in progressive science. I think instead that understanding the different kinds of assumptions in play in mathematical modelling within the various `higher' realms of the scientific enterprise can help us demarcate pseudoscientific modelling---that is, mathematical models that are more akin to novels or films---from mathematical models of complex systems that, whilst necessarily fictional in the sense that they only trace some of the core emergent dependencies or patterns inherent in the target system, are realistic in the sense that, by foregrounding some of the right emergent dependencies (i.e. by embodying a good theory of the target system), have locked into the “deep structural forces" of the target system that determine its evolution through time. As we'll see, I think that this can be illustrated by case studies: by drawing a contrast between---on the one hand---weather/climate modelling and successful modelling in ecology (population dynamics) (and various work on the margins of economics (complexity economics) and the new field of cliodynamics), and---on the other hand---DSGE-modelling in macroeconomics, whose predictive failures can be explained by the fact that the entire enterprise instantiates false domain assumptions (fundamentally embodying the mistaken assumption that the macroeconomy is an equilibrium system).

Before we can analyse these case studies, it is necessary to introduce an analytical tool: a taxonomy of assumptions in scientific theories due to the philosopher Alan Musgrave (contained in his brief discussion of Milton Friedman's very confused philosophy of instrumentalism from the in influential essay “Methodology of Positive Economics"). Musgrave claims that there are three kinds of assumptions worth distinguishing: negligibility assumptions, domain assumptions and heuristic assumptions. The third is less relevant for our investigations, but I shall quickly describe the first two. A negligibility assumption is an assumption that a certain phenomenon not known to be negligible in the real world (or known to be not negligible at least some of the time) is negligible. Galileo tested his theory about the acceleration of falling bodies in a vacuum in the real world with the assumption that (in his experimental cases) air-resistance was of no consequence (correct, in his experimental cases, though clearly not in general). In the complex-systems-level sciences,

physics" [Feyman 1985, 4]) has been confirmed by predictions at an astonishing level of precision---levels of precision that give us reason to think that the equations we have come up really do describe the fundamental dynamics about as well as one could imagine describing them.

⁴ Godfrey-Smith connects this explanation to the doctrine of Structuralism in philosophy of mathematics, the essence of which is given away by the title of Michael Resnik's 1997 book, Mathematics as the Science of Patterns.

we might consider calling the equivalent assumption-category the relative insignificance assumption, simply because in modelling population dynamics in ecology, financial crises in economics, or the rise and fall of social cohesion in history, a scientist will be forced to leave out plenty of variables which they know to make a super-negligible difference much or all of the time in the real world. This, of course, does not mean that they cannot do proper science: as I'll argue when I introduce the case studies, the scientist maintains a claim to doing science for a complex system if they can generate system state-changes that mirror the dynamics of the target system from the simplest possible model embodying their given theory, because this is a good indication that they have struck causal realism. The second kind of assumption, the domain assumption, is of a rather more general kind: it can be thought of as any kind of positive assumption that specifies the world embodied by the model (as opposed to the real world). More helpfully, an unrealistic domain assumption is an aspect of the mathematical structure of a model that is most definitely at odds with the target system it is supposed to model. I see the core instrumentalist mistake as the elision of the fact that any successful predictions made by a model which embodies too many unrealistic domain assumptions will be entirely accidental (and the addition of parameters and extra variables, bells and whistles, to bootstrap predictive power will just make the situation worse when there is a significant shift in the state of the system).

We now have the framework necessary for our case studies, beginning with ecology itself, or, more precisely, “population dynamics". In his 2002 treatise on the methodology, theory and models of mathematical ecology, Complex Population Dynamics, the ecologist and `cliodynamicist' Peter Turchin argues that ecology has become a mature, predictive science, and he attempts to demonstrate this via an analysis of a series of models. Relevant to this enterprise, the book also contains (and embodies) an expression of Turchin's modelling philosophy for complex systems science, a philosophy he has applied in his research in both ecology and on human societies in his development of the Structural-Demographic Theory of the internal cycles of empires. I think this modelling philosophy gives us the key to understanding what good complex-systems modelling should look like. Turchin's methodological algorithm goes as follows.

First, if there are two or more theories postulating fundamentally distinct (though not necessarily completely incompatible) causal mechanisms for a certain explanandum, then these theories must be given a maximally simple mathematical expression and their predictions/retrodictions compared with the data (maximally being the key word, of course, since `simple' in this context does not mean that they are easy-to-solve differential equations like the models introduced to beginning ecology students, or that they are as blatantly unrealistic as such models are). ⁵ Two miscellaneous examples of such explananda due to Turchin himself (from history and ecology respectively) are:

a)      The `saw-toothed' cohesion-cycles observed throughout historical societies, for which the two general competing theories are the “Malthusian theory" and the “Structural-Demographic (`elite overproduction') theory" (he has written several books explicating the latter).

b)   The fluctuations in “southern pine beetle" population size, for which the dominant explanation in the 1980s was climactic (exogenous), but which now appears (thanks to analysis of time-series data) to be explicable mostly “by second-order endogenous factors" [Turchin 2003, 164].

Ideally, this procedure should at least allow for a pretty confident falsification of very bad theories---since, if the fit for the simple version is far off the mark, then it seems a very strong bet that the theory is causally unrealistic, and if a relatively simple model generates dynamics that broadly mirror those in the target system (of course this requires a variety of quantitative measures determining the average period, period strength etc), then one has strong evidence that it is a theory worth pursuing. Second, when it comes to deciding between models of varying complexity embodying the same theory (i.e. once one has a strong theory), the procedure is again empirical. As Turchin explains in Complex Population Dynamics, “The question of whether to include population structure or not can be resolved by constructing and contrasting with data

⁵ It should also be noted that whilst the above principle was fairly simple to state, this conceals the sophisticated statis-tical techniques and intelligent quantitative-measure choice that are needed to execute it properly (“We can then attempt to distinguish between the competing models by doing parallel time-series analyses on the data and on model outputs, to obtain quantitative measures of their relative success at matching the patterns in the data."[165]).

two versions of the model: one that explicitly incorporates individual variability [fewer relative-insgificance assumptions], and the other that averages over this variability with judiciously chosen functional forms [more relative-insignificance assumptions]." [Turchin 2003, 65] The long and short of the philosophy is this: (a) that we should add extra parameters and include more specific variables iff these lead to a model which is a better fit to the facts, and (b) that what is of most importance fundamentally is causal realism. Crucially, this methodological algorithm should actively discourage the production of models with clearly false domain assumptions, on account of the strongly counter-instrumentalist first step.

The highly advanced science of metereology may, on the surface, seem to impugn the first principle of this methodological philosophy, inasmuch as there are no serious climate models in contemporary use that could be described as `simple' (or even which are solvable by hand at all). However, this is superficial, since one critical factor that distinguishes metereology from ecology is that, from its very inception in the 1950s, the modellers had the key `theory' in place: fluid dynamics. As the aforementioned Nature article notes, “The Navier-Stokes and mass continuity equations (including the effect of the Earth’s rotation), together with the first law of thermodynamics and the ideal gas law, represent the full set of prognostic equations upon which the change in space and time of wind, pressure, density and temperature is described in the atmosphere" [Bauer et al. 2015, 48], meaning that the first principle described by Turchin was irrelevant. As for Turchin's second principle, I believe it is in play; it just so happens that adding complexity to models and taking advantage of all the computing power available has been the way to generate better predictive power in metereology. ⁶

The contrastive case study is the discipline of macroeconomics. I claim that the predictive uselessness of even the most celebrated macroeconomic models, particularly (but by no means solely) when it comes to major recessions and downturns, and particularly since 2008 [Edge et al. [2010], Keen [2011]] ---a state of a airs which has led to foundational questioning by some elder statesmen of the field [Blanchard [2016], Romer [2016] ]---has evolved from a historically rooted excessive tolerance of unrealistic domain assumptions (i.e. a philosophy of instrumentalism), which is anathema to Turchin's methodology. Economics evolved as a mathematical science before the advent of complex-systems mathematics, much like ecology, but the mainstream of the discipline did not respond to the complex-systems revolution at all. Whilst there was much upheaval within economics in the 1970s and 1980s (with the rise of Friedman and Lucas and the turn to microfounded macroeconomics), this only pushed the discipline in a more unrealistic direction, in terms of the fundamental assumptions about the economy, with the paradigm of the `representative consumer' (nothing like a real person) and `representative firm' (with a cost structure unlike any real firm) optimising towards an infinite horizon [Keen [2011], Lavoie [2015]]. More fundamentally, the central paradigm of mainstream neoclassical macroeconomics is the model of the economy as a system that does return to equilibrium absent “exogenous shocks", which is not path-dependent and which functions essentially like a machine [Keen [2011], Lavoie [2015]]. This is fundamentally opposed to reality, which is why the only predictive successes that DSGE models ever did generate were the result of tailoring models to recent data, so-called “overfitting". This kind of methodological practice which may generate very impressive near-term successes but it is also likely to result in massive scientific crisis when the target system state changes in a dramatic way (as in the GFC), because the overfitted model that was in use up to this state change is now doubly useless: not only has it been shown decisively that it does not embody the causal structure of the target system, there is nothing anyone can do about it, because how are we to know what exactly went wrong in such a highly parametrised, massively complicated, baroque model?

A key difference between ecology and economics can be observed in the way and in which the role of assumptions changes as one progresses in the subject. In ecology, the elementary models taught to students---the three central `laws' (as Turchin calls them [Turchin [2001]]) of the exponential growth model,

⁶ Again, we should add a detail here: metereologists and climate scientists don't generate their predictions just from running one perfect model. Because of the path-dependence of chaotic systems and our extremely imperfect knowledge about all the potentially relevant atmospheric data, it is necessary to use what is called “ensemble-modelling" (comparing similar models with different initial conditions, etc) to generate reliable results.

the logistic growth model and the Lotka-Volterra model---are not used precisely because of their very bad domain assumptions: Turchin calls the most complicated of the three, the Lotka-Volterra equations, “a horribly unrealistic model for real resource-consumer systems [...] so bad that, to my knowledge, there has been no successful application of it to any actual population system, whether in the field or laboratory" [Turchin 2001, 21]. But in economics, unlike ecology, the higher-level models, although of course far more complicated than the elementary models (their mastery requiring years of graduate training), are just as profoundly unrealistic as the elementary ones.

In summary, and in answer to the question, mathematical models in ecology (population dynamics) serve their purpose when they are built in such a way that their relative-insignificance assumptions are orthogonal to the key causal structure of the target system, enabling the models to generate solid predictions even as the state of the target system changes (because they are calibrated to the drivers of the system's state-changes). This fact helps us see how and in what sense assumptions matter in complex-systems science: the chief task has to be to find the right theory, and this task will inevitably result in the mulching of mathematical models that instantiate completely unrealistic domain assumptions. The scourge of `overfitting' can be avoided by focussing on arriving at a model that can generate all the states of the target system, not just generate some neat predictions in a period of equilibrium, and this is equivalent to a causally realistic model.

References

Bauer, Peter, Thorpe, Alan, & Brunet, Gilbert. 2015. “The quiet revolution of numerical weather prediction".

Nature, 525(7567), 47{55.

Blanchard, Olivier. 2016. “Do DSGE Models Have a Future?" Peterson Institute of International Economics, 11{16.

Colyvan, Mark, & Ginzburg, Lev R. 2003. “Laws of nature and laws of ecology". Oikos, 101(3), 649{653.

Edge, Rochelle M., Gurkaynak, Refet S., Reis, Ricardo, & A., Sims Christopher. 2010. “How Useful Are Estimated DSGE Model Forecasts for Central Bankers?" [with Comments and Discussion]. Brookings Papers on Economic Activity, 209{259.

Feyman, Richard P. 1985. QED: The Strange Theory of Light and Matter. Princeton University Press.

Godfrey-Smith, Peter. 2009. “Models and Fictions in Science". Philosophical Studies, 143(1), 101{116.

Keen, Steve. 2011. Debunking economics: the naked emperor dethroned? Rev. and expanded edn. London:

Zed Books Ltd.

Lavoie, Marc. 2015. Post-Keynesian Economics: New Foundations. Edward Elgar.

Lucarini, Valerio. 2013. “Modelling Complexity: The Case of Climate Science". Pages 229{253 of: Ghde, Ulrich, Hartmann, Stephen, & Wolf, Jrn H. (eds), Models, Simulations, and the Reduction of Complexity. De Gruyter.

Musgrave, Alan. 1981. “Unreal Assumptions in Economic Theory: The F-Twist Untwisted". Kyklos, 34(3), 377{387.

Romer, C. D. 2016. The Trouble with Macroeconomics. Commons Memorial Lecture of the Omicron Delta Epsilon Society.

Turchin, Peter. 2001. “Does population ecology have general laws?" Oikos, 94(1), 17{26.

Turchin, Peter. 2003. Complex Population Dynamics: A Theoretical/Empirical Synthesis. Princeton Uni-versity Press.