Economics is an inexact science

When I wrote about why the BBC should treat a clear consensus in economics the same way as it now treated climate science, I got a number of comments about why economics is not a science. A common theme was that economics couldn’t prove theories ‘beyond doubt’ the same way as the hard sciences could. A more sophisticated version of this complaint is that most economic theories cannot be disproved in the same way that Popper thought scientific theories could be disproved.

All this ignores a key feature of any social science, which is their inexact nature. Instead we have accumulations of evidence that confirm the applicability of some theories and reject the applicability of others. Economists’ views about what models are applicable change as this evidence accumulates.

A good example involves the minimum wage, as Noah Smith suggests. The basic economic model suggested even a modest minimum wage should significantly reduce employment, but economists discovered that the evidence did not show this. As this evidence accumulated, alternative theories and models (monopsony and search) were thought to be more relevant. It is this response to evidence that makes economics a science.

Jo Michell writes “The scientific method of forming a hypothesis and then testing that hypothesis against reality can never be the final arbiter of knowledge, as it can in the physical sciences.” He is right that no single experiment or regression can kill a theory, but wrong that the accumulation of evidence is not the final arbiter, because no other arbiter is available. He links to a post by Noah Smith which talks about the failures of forecasting. But as that post makes clear, this is not about data rejecting models, but the inability of models to predict the future. We would never dream of condemning medics because they cannot predict the exact time of our death, still less suggest that this failure indicates they are not doing science.

Of course economics involves cases where economists appear too reluctant to give up their favoured models. You can find similar stories in the hard sciences. There will be more such stories in economics because the inexact nature of economics makes it easier to discount any single piece of evidence. What I cannot understand is what leads someone like Russ Roberts to argue against the use of evidence, and instead that “economics is primarily a way of organizing one’s thinking”. Astrology is also a way of organising one’s thinking, but it fails because evidence does not back it up.

That comparison is slightly unfair, because while the theory behind astrology is obviously implausible, the basic principles of microeconomics are not. In a class on economic methodology I once drew a huge tree that showed how most of economics could be derived from principles of rational choice. But go beyond the basics, and add in complications involving information and transactions costs (to name but two) and you very quickly derive competing models. There is no single model that comes from thinking like an economist, so for that reason alone we need data to tell us which models are more applicable.

So thinking like an economist does not tell me at what point raising the minimum wage will reduce employment. But why would anyone want to keep their models from being proved relevant or otherwise by data? The only reason I can think of is that some models give answers that are ideologically convenient. Of course allowing data to establish the relevance of some models over others does not make economics ideology proof. For example people can always select the one study that suggests that fiscal policy does not influence output and ignore the hundreds that show otherwise. That is why the accumulation of evidence, which includes its replicability, is so important. If you think economics has problems in that respect, have a look at psychology.

This is why economists views about the long term impact of Brexit should be treated as knowledge rather than just an opinion. Here knowledge is shorthand for the accumulation of evidence consistent with plausible theory. Sometimes the theories are common sense, like making trade more difficult will reduce trade. Estimates of the size of trade reduction based on evidence are uncertain, but they are better than estimates based on wishful thinking. Empirical gravity equations consistently show that geography still matters a lot in determining how much is traded. Finally there is clear evidence that trade is positively associated with productivity growth. To say that all this has no more worth than some politicians opinion is ultimately to degrade evidence and the science which interprets it.

Economics Rules by Dani Rodrik

I didn’t want to talk about this book before I had finished it: I somehow think Noah’s contrary approach has its shortcomings! The first and most important thing to say is this is a great book. Not because it gave me some huge new insight or knowledge, although I did learn quite a bit about other parts of economics, but because it had a way of putting things which was illuminating and eminently sensible. Illuminating is I think the right word: seeing my own subject in a new light, which is something that has not happened to me for a long time. There was nothing I could think of where I disagreed (which given the book’s wide scope is quite something), and plenty where the inner blogger in me said I wish I’d written that.

So who should read it? To be honest I cannot think of anyone who should not, as I think most of the material could be understood by interested non-economists. His writing style is enviable - it seems so effortless! (That’s me as blogger again.) The people who should especially read it are those who interact with economics or economists and are either unclear or distrustful about what economists are about (other social scientists particularly).

The first part of the book sets out a way of thinking about economics, and in particular to the models that economists could not live without. The key idea is that there are many valid models, and the goal is to know when they are applicable to the problem in hand. This idea has already attracted some attention, including Noah Smith’s post I linked to earlier.

I must admit when I first read it I thought well of course, doesn’t everyone understand that? I remember way back when I did my undergraduate degree, hearing a lecture from a young David Newbury I think, who said the days of big models (models of everything) were over in economics, and that today economists focused more on small but focused models, looking at particular problems or issues. But then as I read on I began to realise that I typically did not employ this knowledge into how I discuss the subject, which is exactly what Rodrik does. 

One area of economics that you might think this would not apply is macro, but it does. It is routine, for example, to split issues up by time: the famous short, medium and long run. A New Keynesian model is not going to tell you much about long run growth, but a Solow growth model does not tell you much about involuntary unemployment. The point here is not that an all encompassing model could not be built - it could, and sometimes individuals or institutions try to do that - but if it was it would be unwieldy, and we would want to break it up in our minds to understand how it works. (I used a related idea of ‘theoretical deconstruction’ in an EJ paper some time ago.) An important point that follows from that is that although we work with different models, it is important that we know how they interconnect, or at least how they relate to each other.  

Rodrik spends a good part of the book describing how you ‘navigate among models’. He warns that these methods are as much a craft as a science. Many have picked up on that, presuming that this is something that a proper science would not do. But as I have often said, the best analogies for economics are with medicine rather than physics. When a doctor diagnoses an illness based on symptoms, they could also be said to be using craft rather than science.

Let me give you a simple example from macro. How do we know if most economic cycles are described by Real Business Cycles (RBC) or Keynesian dynamics. One big clue is layoffs: if employment is falling because workers are choosing not to work we could have an RBC mechanism, but if workers are being laid off (and are deeply unhappy about it) this is more characteristic of a Keynesian downturn. This simple test beats any amount of formal econometric comparison. Craft maybe, but not a very difficult craft in this case.    

Lots of people get hung up on the assumptions behind models: are they true or false, etc. An analogy I had not seen before but which I think is very illuminating is with experiments. Models are like experiments. Experiments are designed to abstract from all kinds of features of the real world, to focus on a particular process or mechanism (or set of the same). The assumptions of models are designed to do the same thing.

Although I found that Rodrik’s discussion of how you select the right model familiar and sensible, it remains vague in the philosophical sense, as Emrah Aydinonat points out. But he also finds them instructive, so they are a work in progress that hopefully philosophers and economists can interact on. (It is worth passing on a point which Aydinonat makes, which is that unlike many economists who write about methodology, Rodrik has read the relevant literature!) Thinking about alternative models that differ in their applicability to particular problems is certainly a more insightful approach than the kind of Popperian stuff that most economists remember.

If this makes the book sound like a philosophical tome, that is quite wrong. It is a very readable account of how economists do what they do: the philosophical grounding is there but it is not intrusive, and instead the book focuses on practical examples. What Rodrik then does with this perspective of many models is to think about a lot of the issues outsiders have about economists: how ideological are they, for example. Towards the end he discusses what went wrong in the financial crisis. Once again the perspective is illuminating: there were for sure models that said a crisis should not happen, but also plenty of models around at the time that explained pretty well why it could. The mistake many economists made was to choose the wrong models, and he discusses why that might have happened. This perspective shows why a simple ‘the crisis shows economics must be flawed’ misses the point.

Hopefully that is enough to make you read this book.  

Encouraging dialogue between economists and social scientists

In a way this is a rather trivial post, about language and attitudes as much as anything, that follows from some of the reaction to this post and related debates. One big difference between most (not all) mainstream economists compared to their heterodox or other social science colleagues is insularity. Political scientists will talk to sociologists who in turn talk to international relations people as a matter of routine. Economists by and large talk to each other. This is not because their subject matter is narrow - economists are notorious for applying their tools way beyond economics.

Most of the time I think that is fine, but sometimes it is not. How do we deal with the times that are not? I want you - as either an economist or social scientist or interested spectator - to think about a visit to your doctor. Why? Because economists should really think of themselves like doctors. (I know some want to think of themselves as physicists, but what can we do.) They are trying to understand highly complex and erratic systems based on a small number of principles, where most of the time they have very little idea of what is going on. But they have data, and some of the time they can make a lot of difference to people’s lives.

Now supposing your doctor prescribes you a course of treatment. It involves drugs that you have read a little about, and you have some concerns. If you were a social scientist would you say to your GP something like the following:

“I’m a little worried about this. I feel that you may not have adequately addressed the ontological and epistemological issues that are raised here. What exactly is ‘treatment’, and when is it necessary? Have you thought about the complex social and economic interrelationships that lay behind your prescription? Is the nature of what you call ‘illness’ really independent of the nexus of interactions that could be loosely called the medical profession?”

I kind of hope you wouldn’t, because I do not think you would get very far if you did. You might be much better off asking something like the following.

“I’m a little worried about this. Have you thought about whether this treatment is appropriate to my particular circumstances (HT Ben Goldacre), or are you reacting to pressure from the drug company or someone else? Has that company published all its trial data, or only the trials that were successful, and how was success defined in this case. Is this really going to make me better, or just increase someone’s profits?”

It is a simple and obvious (I did say trivial) point - you will get much further if you talk specifics in a language your doctor will understand, rather than in generalities and terminology they will not. Economists want (or need) to know why their approach is missing key issues or linkages which compromise their analysis, just as the doctor needs to know why they might be recommending the wrong treatment. You would not insist that your doctor needed to have studied economics before they can be a good doctor.

But if you were an economist, would you think it legitimate for your GP to respond like this.

“How inappropriate of you to ask me these questions. I’m a doctor, and I know from my years of knowledge and experience what is the right treatment for you. As you cannot know what I know, then you should not get involved in these issues. Some of the things you mention are really none of my business, and I do not see why I should worry about them.”

Now as an economist you know that such a response from your doctor would be both arrogant and naive. The doctor should ask about the quality and objectivity of the information they receive, and know full well that drug companies exist to make money. But might your response to a social scientist be as arrogant and naive?

Let me take a real world economic problem: the response to the financial crisis. Some have suggested that banks have become too large and need to be broken up, or that the activities of high street banking need to be separated from the activities of the casino. Your economic analysis tells you that networks of many small entities can be as subject to crises as networks involving a few large banks. You are also able to devise a system of Chinese walls that mean that the activities of the casino can be separated from those of the high street even within the same company, and your political masters seem to prefer this approach. You recognise that different assets differ in their liquidity, and so you devise complex weighting algorithms for computing capital ratios. Your suggestions form the basis of negotiations between officials and bankers, and a set of rules and regulations are agreed.

Over the next few years you watch in dismay as your complex system begins to unravel. The CEOs of the large banks seem to constantly have the ear of politicians, who in turn gradually compromise your elaborate controls to render them less and less effective. Those in charge of administering the rules find it much more lucrative to work for the banks, and so regulators gradually lose expertise and resolve.

And you realise that right from the start you made the wrong choice. You decided to focus on what you knew, which was how to design systems that worked well as long as those systems remained unchanged, but which were not robust to intervention by self-interested parties. In short, they were too open to rent-seeking. You realise that actually the best thing to have done was to break up the banks so that their political power was forever diminished. And you recall a conversation with your social science colleague when this all started, who might have been trying to tell you this if only you had understood the words he was using.