Forecast errors compared

And a coda defends experts against Aditya Chakrabortty

A recent conversation got me thinking about different types of macroeconomic forecast error, and what implications they might have for macroeconomics. I’ll take three, from a UK perspective although the implications go well beyond. The errors are the financial crisis, the lack of a downturn immediately after Brexit, and flat UK productivity.

The immediate cause of the Global Financial Crisis (GFC) was the US housing market crash, but that alone should have caused some kind of downturn in the US, with limited implications for the rest of the world. What caused the GFC was the lack of resilience of banks around the world to a shock of this kind.

Were there any indications of this lack of resilience? Here is an OECD series for banking sector leverage in the UK: the ratio of bank assets to capital. The higher the number, the more fragile banks are becoming.

UK Banking sector leverage: Source OECD

The first and perhaps most important problem with forecasting the financial crisis was that macro forecasters were not looking at data like this. For most it was not on their radar, because banks, let alone bank leverage, played no role in their models. It was a sin of omission, a big gap in our macro understanding. (Whether, if forecasters had been having to forecast this data, they would have predicted a crisis is improbable, but some would have at least noted it as an issue.)

Moving on to the second mistake, it is often said (correctly) that forecasters are very bad at predicting turning points or dramatic changes. But many did predict such a change immediately following the Brexit vote: a sharp and immediate slowdown in demand caused by the uncertainty of Brexit. It didn’t happen. The main reason was consumption, which held up by more than people were expecting, given the fall in real incomes that was likely to come from the Brexit depreciation. There are two and a half obvious explanations for this. First, because of Leave propaganda half the population thought Brexit would make them at least no worse off. Second, those who did anticipate the rise in import prices may have taken the opportunity to buy consumer durables made overseas to beat the prospective price increase. The half is that the Bank cut interest rates a bit.

None of these effects are very new. They may not have been incorporated into the forecasters’ models, but they could in principle have been incorporated using the forecaster’s judgement, although getting the quantification right would have been very difficult. In the end we got the slowdown, but delayed until the first half of this year, as Leavers began to face reality and the higher import prices came through, so it was an error of timing more than anything else (although it was apparently enough to make MP Liz Truss change her mind and support Brexit!). You could describe it as an unchallenging error, because it could easily be explained using existing ideas. It is the kind of error that forecasters make all the time, and which makes forecasting so inaccurate.

The third error was UK productivity, which I talked about at length here. Until the GFC, macro forecasters in the UK had not had to think about technical progress and how it became embodied in improvements in labour productivity, because the trend seemed remarkably stable. So when UK productivity growth appeared to come to a halt after the GFC, forecasters were largely in the dark. What many like the OBR did, which is to assume that previous trend growth would quickly resume, was not the extreme that some people suggest. It was instead a compromise between continuing no growth and reverting to the previous trend line, the second being what had happened in previous recessions.

My point of writing about this again is that I think this third error is much more like the GFC mistake than the post-Brexit vote mistake. In both cases something important that forecasters were used to taking for granted started behaving in a way that had not happened since WWII. Standard models were used to treating technical progress as an unpredictable random process. Now it is just possible that this is still the case, and the absence of technical progress in the UK and to a lesser extent elsewhere is just one of those things that will never be explained. But for the UK at least the coincidence with the GFC, austerity and now Brexit seems too great. As as I showed in the earlier post growth has not been exactly zero but has oscillated in a way that could be related to macro events.

If there is some connection, both in the UK and elsewhere, between the decline in economic productivity growth and macroeconomic developments, then this suggests an important missing element in macromodels. And like the financial sector, there is an existing body of research that economists can draw on, which is endogenous growth theory. There are examples of that happening already.

But I want to end with a plea. After the financial crisis too many people who should have know better said that failing to predict the financial crisis meant that all existing mainstream macroeconomics was flawed. It was rubbish, but such attitudes did not help when some of us were arguing against austerity on the basis of standard macroeconomic ideas and evidence. Now with UK productivity, we have Aditya Chakrabortty saying that experts at the OBR “are guilty of a similar un-realism and they have proven just as impervious to criticism” as people like Boris Johnson or Liam Fox. Not content with this nonsense, he says “This age of impossibilism is partly their creation”.

This is just wrong. Look at the elements of neoliberal overreach. Economists didn’t start calling for tight immigration controls and using immigrants as a scapegoat for almost everything. Most academic economists did not call for austerity. Almost all economists did not want to get rid of our trade agreements with the EU. Even if economists had warned about the financial crisis they would have been ignored because of the political power of finance. If all economists had thought productivity would continue flat we would have just had more austerity. [1] And in making this basic mistake, it is ironically Aditya Chakrabortty who has joined Michael Gove and other Brexiteers in having had enough of experts.

[1] Less expected productivity growth means lower future output which means lower future tax receipts which means, given the government’s austerity policy, more cuts in public spending.

Attacking economics is a diversionary tactic

Forgive the numbered note form. For some reason it seems appropriate to me in this case

1. The financial crisis in the UK was the result of losses by banks on overseas assets, originating from the collapse in the US subprime market. It was not a result of excessive borrowing by UK consumers, firms or our government. As the Bank’s Ben Broadbent points out, “Thanks to the international exposure of its banks the UK has been, in some sense, a “net importer” of the financial crisis.” This overseas lending caused a crisis because banks were far too highly levered, and so could not absorb these losses and had to be bailed out by the government.
   
   
2. This is why UK macroeconomists failed to pick up the impending crisis. They did routinely monitor personal, corporate and government borrowing, but not the amount of bank leverage. Macroeconomists generally acknowledge that they were at fault in ignoring the crucial role that financial sector leverage can play in influencing the macroeconomy. There has been a huge increase in the amount of research on these finance-macro linkages since the crisis.
   
   
3. But supposing economists had ensured that they knew about the increase in bank leverage and had collectively warned of the dangers of excessive risk taking that this represented. Would it have made any difference? There are good reasons for thinking it would not.
   
   
4. The main evidence for this is what has happened after the crisis. Admati and Hellweg have written persuasively that we need a huge increase in bank capital requirements to bring the ‘too big to fail’ problem to an end and avoid a future banking crisis, and the work of David Miles in the UK has a similar message. I have not come across an academic economist who seriously dissents from this analysis, but it has no impact on policy at all. The power of the banking lobby is just too strong.
   
   
5. So the response of economists to the financial crisis has been as it should be. The error in neglecting bank leverage is being addressed. Economists have come up with clear proposals about how to avoid the crisis happening again. And these proposals have been pretty well ignored.
   
   
6. In terms of conventional monetary and fiscal policy, academic economists got the response to the crisis right, and policymakers got it very wrong. Central banks, full of economists, relaxed monetary policy to its full extent. They created additional money, rightly ignoring those who said it would bring rapid inflation. Many economists, almost certainly a majority, supported fiscal stimulus for as long as interest rates were stuck at their lower bound, were ignored by policymakers in 2010, and have again been proved right.
   
   
7. So given all this, why do some continue to attack economists? On the left there are heterodox economists who want nothing less than revolution, the overthrow of mainstream economics. It is the same revolution that their counterparts were saying was about to happen in the early 1970s when I learnt my first economics. They want people to believe that the bowdlerised version of economics used by neoliberals to support their ideology is in fact mainstream economics.
   
   
8. The right on the other hand is uncomfortable when evidence based economics conflicts with their politics. Their response is to attack economists. This is not a new phenomenon, as I showed in connection with the famous letter from 364 economists. With austerity they cherry picked the minority of economists who supported it, and then implemented a policy that even some of them would have disagreed with. (Rogoff did not support the cuts in public investment in 2010/11 which did most of the damage to the UK economy.) The media did the rest of the job for them by hardly ever talking about the majority of economists who did not support austerity.
   
   
9. The economic costs of Brexit is just the latest example. Critics have focused on the most uncertain and least important predictions about Brexit, made only by a few, to attack all Brexit analysis. The fact that this prediction involved an unconditional macro forecast, while the assessment made by a number of groups about the long term cost involves a conditional projection based largely on trade equations, seems to have completely escaped the critics. More important, the fact that the predicted depreciation in sterling happened, and is in the process of already causing a large drop in living standards, is completely ignored by these critics.
   
   
10. Attacking economists over Brexit is designed to discredit those who point out awkward and uncomfortable truths. Continuing to attack economists over not predicting the financial crisis, but failing to ignore their successes, has the effect of distracting people from the group who actually caused this crisis, and the fact that very little has been done to prevent a similar crisis happening in the future.
    

Miles on Haldane on Economics in Crises

Anything that says economics is in crisis always gets a lot of attention, particularly after Brexit (because economists are so pessimistic about its outcome), and Andy Haldane’s public comments were no exception. But former Monetary Policy Committee colleague David Miles has hit back, saying Haldane is wrong and economics is not in crisis. David is right, but (perhaps inevitably) he slightly overstates his case.

First an obvious point that is beyond dispute. Economics is much more than macroeconomics and finance. Look at an economics department, and you will typically find less than 20% are macroeconomists, and in some departments there can be just a single macroeconomist. Those working on labour economics, experimental economics, behavioural economics, public economics, microeconomic theory and applied microeconomics, econometric theory, industrial economics and so on would not have felt their sub-discipline was remotely challenged by the financial crisis.

David Miles is also right that economists have not found it difficult to explain the basic story of the financial crisis from the tools that they already had at their disposal. Here I will tell again a story about an ESRC seminar held at the Bank of England about whether other subjects like the physical sciences could tell economists anything useful post-crisis. It was by invitation only, Andy Haldane was there throughout, and for some reason I was there and asked to give my impressions at the end. In the background document there was a picture a bit like this.

UK Bank leverage: ratio of total assets to shareholder claims. (Source Bank of England Financial Stability Report June 2012) Added by popular request 17/1/17 [3]: 

I made what I hope is a correct observation. Show most economists a version of this chart just before the crisis, and they would have become very concerned. Some might have had their concern reduced by assurances and stories about how new risk management techniques made the huge increase in leverage seen in the years just before the crisis perfectly safe, but I think most would not. In particular, many macroeconomists would have said what about systemic risk?

The problem before the financial crisis was that hardly anyone looked at this data. There is one institution that surely would have looked at this like this data, and that was the Bank of England. As Peter Doyle writes:

“ .. it was not “economics” that missed the GFC, but, dare I say it (and amongst some others), the Bank of England.”

If there is a discussion of the increase in bank leverage and the consequent risks to the economy in any Inflation Reports in 2006 and 2007 I missed it. I do not think we have been given a real account of why the Bank missed what was going on: who looked at the data, who discussed it etc. I think we should know, if only for history’s sake.

What I think David Miles could have said but didn’t is that macroeconomists were at fault in taking the financial sector for granted, and therefore typically not including key finance to real interactions in their models. [1] As a result, the crisis has inspired a wave of new research that tries to make up for that, but this involves using existing ideas and applying them to macroeconomic models. There has also been new work using new techniques that has tried to look at network effects, which Andy Haldane mentions here. Whether this work could be usefully applied much more widely, as he suggests, is not yet clear, and to say that until that happens there is a crisis in economics is just silly.

The failure to forecast that consumers after the Brexit vote would reduce their savings ratio is a typical kind of forecasting error. Would they have done this anyway, and if not what about the Brexit vote and its aftermath inspired it, we will probably never know for sure. This kind of mistake happens all the time in macro forecasting, which is why comparisons to weather forecasting and Michael Fish are not really apt. [2] That is what David Miles means by saying it is a non-event.

What is hardly ever said, so I make no apologies for doing so once more, is that macroeconomic theory has in some ways ‘had a good crisis’. Basic Keynesian macroeconomic theory says you don’t worry about borrowing in a recession because interest rates will not rise, and they have not. New Keynesian theory says creating loads of new money will not lead to runaway inflation and it has not. Above all else, macroeconomic theory and most evidence said that the turn to austerity in 2010 would delay or weaken the recovery and that is exactly what happened. As Paul Krugman often says, it is quite rare for macroeconomics to be so fundamentally tested, and it passed that test. We should be talking not about a phoney crisis in economics, but why policy makers today have ignored economics, and thereby lost their citizens' the equivalent of a lot of money.

[1] In the COMPACT model I built in the early 1990s, credit conditions played an important role in consumption decisions, reflecting the work of John Muellbauer. But as I set out here, proposals to continue the model and develop further financial/real linkages were rejected by economists and the ESRC because it was not a DSGE model.

[2] Weather forecasts for the next few days are more accurate than macro forecasts, although perhaps longer term forecasts are more comparable. But more fundamentally, while the weather is a highly complex system like the economy. It is made up of physical processes that are predictable in a way human behaviour will never be. As a result, I doubt that simply having more data will have much impact on the ability to forecast the economy.

[3] Total asset are the size of the bank's balance sheet. Shareholder claims are the part of those assets that belong to shareholder, and which therefore represent a cushion that can absorb losses without the bank facing bankruptcy. So at the peak of the financial crisis, banks had over 60 times as many assets as that cushion. That makes a bank very vulnerable to loss on those assets.

Banks, economists and politicians: just follow the money

Economics rightly comes in for a lot of stick for failing to appreciate the possibility of a financial crash before 2007/8. However it is important to ask whether things would have been very different if it had. What has happened to financial regulation after the crash is a clear indication that it would have made very little difference.

There is one simple and straightforward measure that would go a long way to avoiding another global financial crisis, and that is to substantially increase the proportion of bank equity that banks are obliged to hold. This point is put forcibly, and in plain language, in a recent book by Admati and Hellwig: The Bankers New Clothes. (Here is a short NYT piece by Admati.) Admati and Hellwig suggest the proportion of the balance sheet that is backed by equity should be something like 25%, and other estimates for the optimal amount of bank equity come up with similar numbers. The numbers that regulators are intending to impose post-crisis are tiny in comparison.

It is worth quoting the first paragraph of a FT review by Martin Wolf of their book:

“The UK’s Independent Commission on Banking, of which I was a member, made a modest proposal: the proportion of the balance sheet of UK retail banks that has to be funded by equity, instead of debt, should be raised to 4 per cent. This would be just a percentage point above the figure suggested by the Basel Committee on Banking Supervision. The government rejected this, because of lobbying by the banks.”

Why are banks so reluctant to raise more equity capital? One reason is tax breaks that make finance using borrowing cheaper. But non-financial companies, that also have a choice between raising equity and borrowing to finance investment, typically use much more equity capital and less borrowing. If things go wrong, you can reduce dividends, but you still have to pay interest, so companies limit the amount of borrowing they do to reduce the risk of bankruptcy. But large banks are famously too big to fail. So someone else takes care of the bankruptcy risk - you and me. We effectively guarantee the borrowing that banks do. (If this is not clear, read chapter 9 of the book here.  The authors make a nice analogy with a rich aunt who offers to always guarantee your mortgage.)

The state guarantee is a huge, and ongoing, public subsidy to the banking sector. For large banks, it is of the same order of magnitude as the profits they make. We know where a large proportion of the profits go - into bonuses for those who work in those banks. The larger is the amount of equity capital that banks are forced to hold, the more the holders of that equity bear the cost of bank failure, and the less is the public subsidy. Seen in this way it becomes obvious why banks do not want to hold more equity capital - they rather like being subsidised by the state, so that the state can contribute to their bonuses. (Existing equity holders will also resist increasing equity capital, for reasons Carola Binder summarises based on the work of Admati and Hellwig and coauthors.)

This is why the argument is largely a no brainer for economists. [1] Most economists are instinctively against state subsidies, unless there are obvious externalities which they are countering. With banks the subsidy is not just an unwarranted transfer of resources, but it is also distorting the incentives for bankers to take risk, as we found out in 2007/8. Bankers make money when the risk pays off, and get bailed out by governments when it does not.

So why are economists being ignored by politicians? It is hardly because banks are popular with the public. The scale of the banking sector’s misdemeanours is incredible, as John Lanchester sets out here. I suspect many will think that banks are being treated lightly because politicians are concerned about choking off the recovery. Yet the argument that banks often make - holding equity capital represents money that is ‘tied up’ and so cannot be lent to firms and consumers - is simply nonsense. A more respectable argument is that holding much more equity capital would translate into greater costs for bank borrowers, but David Miles suggests the size of this effect would not be large. (See also Simon Johnson here, John Plender here and Thomas Hoenig here.) In any case, public subsidies are bound to be passed on to some extent, but that does not justify them. Politicians are busy trying to phase out public subsidies elsewhere, so why are banks so different?

There is one simple explanation. The power of the banking lobby (and the financial industry more generally) is immense, from campaign contributions to regulatory capture of various kinds. It would be nice to imagine that the UK was less vulnerable than the US in this respect, but there are good reasons to think otherwise. [2] As a result, the power and influence of banks and bankers within government has hardly suffered as a result of the Great Recession that they played a large part in creating.

So to return to my original question, would it really have made much difference if more mainstream economists had been fretting about the position of the financial sector before the crisis? I think they would have been ignored then even more than they are being ignored now. The single most effective way of avoiding another financial crisis is to reduce the political influence of the banking sector.      

[1] The optimum amount of equity is not 100%, in part because some (subsidised) borrowing does increase discipline on bankers. For a good discussion of other measures that might reduce the too big to fail problem, see this speech from Andy Haldane. An alternative to the state picking up the bill is to inflict losses on depositors, but the economic problems with this are pretty obvious. Nicolas Véron discusses the difficulties the Eurozone has got itself into with this following the Cyprus crash: see also Simon Johnson here.

[2] In Europe, we had what Mark Blyth describes as the biggest bait-and-switch operation in modern history, where a banking crisis involving private sector debts was turned into a public sector debt crisis. While I would be the last person to defend the macroeconomics status quo, I also think there is an element of bait and switch in the ‘macroeconomics in crisis’ idea. Macroeconomic theory tells us a lot of useful things about how to get out of the recession, if only politicians would take some notice.  

Was the financial crisis the fault of DSGE modelling?

I am, like many, in awe of Bank of England director and economist Andy Haldane. However I did wince a bit at his recent Vox piece. He looks at the extent economists are to blame for the financial crisis, and he makes two interrelated claims. Having noted that central banks have traditionally been concerned with the “interplay of bank money and credit and the wider economy”, he suggests that this changed in the decade or so before the crisis. He then says

“Two developments – one academic, one policy-related – appear to have been responsible for this surprising memory loss. The first was the emergence of micro-founded dynamic stochastic general equilibrium (DGSE) models in economics. Because these models were built on real-business-cycle foundations, financial factors (asset prices, money and credit) played distinctly second fiddle, if they played a role at all. 

The second was an accompanying neglect for aggregate money and credit conditions in the construction of public policy frameworks. Inflation targeting assumed primacy as a monetary policy framework, with little role for commercial banks' balance sheets as either an end or an intermediate objective. And regulation of financial firms was in many cases taken out of the hands of central banks and delegated to separate supervisory agencies with an institution-specific, non-monetary focus.”

There is obviously some truth in this, but are these really major factors behind the financial crisis? Imagine looking at the following chart in 2005 or 2006. The increase in leverage that began in 2000 is both dramatic and unprecedented. (Much the same is true for the US.) Was this ignored because central bankers said this variable is not in their DSGE models? In my experience those involved in monetary policy look at a vast amount of information, particularly on the financial side, even though none of it appears in standard DSGE models, and even though their ultimate target might be inflation. For some reason monetary policy makers discounted the risks this explosion in leverage posed, or felt for some reason unable to warn others about it, but I very much doubt these reasons had anything to do with DSGE models.

UK Bank Leverage. Source: Bank of England Financial Stability Report June 2012

           

I say this because to place too much weight on the culpability of DSGE models and inflation targeting can lead to overreaction, and may sideline more fundamental issues. (I don’t, by the way, think Haldane himself falls into either trap: see this interview for example.) Let me take overreaction first. It is one thing to claim, as I have, that the microfoundations approach embodied in DSGE models encouraged macroeconomists to avoid modelling difficult (from that perspective) issues like the role of financial institutions in credit provision. It is quite another to suggest, as some do, that DSGE models are incapable of doing this. This second claim was false before the crisis (e.g. Bernanke, Gertler & Gilchrist, 1999), and has clearly been shown to be false by the post crisis explosion of DSGE work on financial frictions. Forming a rough consensus around a reasonably simple and tractable model of the crisis that can also assess the subsequent policy response will not happen overnight (it never does), and I suspect it will involve tricks which microfoundation purists will complain about, but I’m pretty certain it will happen.

Andy Haldane talks about the need to model the interconnections (networks) of actors and institutions in order to understand how sudden crises can emerge. This must be right, and recent work[1] that begins to do this looks very interesting. However what seems to me critical in avoiding future crises is to understand why leverage increased (and was allowed to increase) in the first place, rather than the specifics of how it unravelled. As I suggested here, we may find more revealing answers by thinking about the political economy of how banks influenced regulations and regulators, rather than by thinking about the dynamics of networks. We should also look at the incentives within banks, and why short term behaviour in the financial sector may be increasing, as Haldane himself has suggested. Investigating networks is clearly interesting, important and should be pursued, but other avenues involving perhaps more conventional economics and political economy may turn out to be at least as informative in understanding how the crisis was allowed to develop.

Postscript

I wrote this before reading this from Diane Coyle, which is well worth reading if you think all microeconomists must be in favour of DSGE. We both went to the conference she mentions, and my own rather different reactions to it partly inspired my post. I'd like to say more about this quite soon.

---

[1] See, for example P Gai, A Haldane and S Kapadia, Journal of Monetary Economics, Vol. 58, Issue 5, pages 453-470, 2011, and other recent work by Kapadia.