How history repeats itself – republished from the IMSL library of 2 November 2012:
Earlier today the Bank of England published three reviews into its recent performance. They contain some interesting insight into how and why analysis sometimes goes awry.
Here are some highlights, along with commentary.
It’s a riveting read. Honest it is.
“Policy makers struggle together with difficult economic issues in an effort to reach consensus on the forecast. Beyond the deliberative value of that collective effort, the resulting consensus forecast has the virtue of simplifying the message of the MPC. Nevertheless, the financial crisis highlighted just how wrong the consensus view can be at times. There could be potential gains from encouraging and considering more fully views that diverge from the consensus.”
- Analysis isn’t a democratic process – just because most people agree with you, it doesn’t mean that you’re right.
- There’s always a trade-off to be made between simplicity and accuracy.
“…the Bank is a centralised and hierarchical organisation with a large decision-making burden residing with the Governor and senior management. Less senior staff undertake analysis of a wide range of policy options, and are often willing to challenge their superiors. But there appears to be some tendency for them to filter recommendations in such a way as to maximise the likelihood that senior staff will find the recommendation palatable. While this makes it easier for the Governor, as ultimate decision-maker, to reach conclusions it risks reducing the range of views he sees and, as such, might lead to a less effective overall outcome.”
- The structure and culture of your organisation will affect the nature of the analysis you produce.
- There’s no point employing brilliant analysts if you don’t listen to them.
- It’s not good enough to be a superb analyst – you have to have the gumption to tell the Governor of the Bank of England he’s wrong.
“The forecast and the analysis offered in support of the forecast should provide for as transparent an engagement as possible with the public, allowing the public to better understand the central bank’s thinking about the evolving outlook and the factors that might cause that outlook to change. Transparency about the forecast also provides an opportunity for those outside the central bank to critique the analysis of the central bank. That feedback can be helpful to the central bank in gaining alternative perspectives on difficult issues of economic analysis.”
- If you don’t show your workings, nobody can identify and challenge the assumptions that have been made, or offer alternative hypothesis.
- Allowing others to challenge your analysis makes for better, more robust assessments.
None of this is intended as criticism of the BoE, or the authors of the reviews. The fact that the Bank agreed to conduct and then publish these independent reviews is indicative of a fundamentally healthy attitude toward results analysis and shows a genuine desire to improve their analysis.
For the sake of clarity I should probably make it clear that my experience of financial analysis extends to periodically staring in disbelief at distinctly unhealthy bank statements, nothing more.