What is social investment analysis?

This is a repost article originally published on pundit.co.nz. He is here with permission.

A few terms that have come into vogue recently are “cost-benefit analysis” (CBA) and “social investment analysis” (SIA), usually proposed by people who have never done either one. They sound good, but they have their limitations. Provided their limitations are understood, they can help you make better decisions. Too often they are ignored and the resulting analysis is of little value, merely reinforcing the conclusion that the decision-maker who commissioned the analysis intended to draw anyway.

Cost-benefit analyzes entered public policy in the 1960s, assessing returns on irrigation projects. (Locals, including their MPs, were very interested in the projects because they involved large government subsidies). In the early 1970s, the National Roads Authority used them to rank road expenses. Ten years later, they played a key role in the debate on major projects (“Think Big”). They were necessary at the time because the projects depended on government grants and other types of public support.

The debate was traumatic; even the economists lost their cool. Part of the problem was that we had to develop the analytical framework as we went along. (The only New Zealand textbook we had was on irrigation projects – I’m not making this up.) We economists made mistakes, but any economic advice was generally ignored by the politicians, who went ahead with even some of the notorious “dogs.” (Roger Douglas admitted that he made bad decisions – against advice – in the New Zealand Steel case.)

The policy lesson was that government should not subsidize businesses, which explains many of the more sensible changes under Rogernomics. Unfortunately, politicians are still interested in incentives for political reasons, and sometimes government involvement is unavoidable – as with broadband rollout and the current debate on inter-island ferry services.

In addition, the government is also involved in many decisions that are not strictly business. The social area exists as a matter of public policy because it cannot be commercialized.

CBA/SIA is most widely used in the health sector. However, it should be noted that politicians have once again tried to reject careful analysis, for example by announcing that they will finance certain pharmaceutical products. (The economic question is whether the same resources could be spent more effectively – with better health outcomes – on other treatments; we don’t know until careful analysis is done.) Applying these techniques to health is fraught with difficulties. I know that after working on many evaluations in many areas and being a member of an international working group that produced a manual on the use of techniques to analyze the use and abuse of psychoactive substances (published by WHO).

The need for a handbook was underscored by the strange debate over the recently published research on the social costs of alcohol, which failed to acknowledge the issues that the working group was working on and did not understand, nor did its critics.

The government has announced that it now intends to apply social investment analysis to areas such as education, justice and social services. It even created a Social Investment Agency to manage its use.

The idea was born during the Key-English government and was implemented during the Ardern-Hipkins government as the Social Care Agency. Despite all the fanfare, sentiment, promises and spending, few results have shown SIA’s effectiveness.

Instructively, there is no published manual for conducting SIA. The reluctance to learn from past experiences or other disciplines is illustrated by various authors Social investment: New Zealand’s policy experiment. They were so devoid of practical experience that the joke is that a consultant is someone who knows 69 ways to make love but has never kissed a girl.

Will the experiment succeed? Experience is that its success, if any, will be limited and almost non-existent if there is no instruction. There will be a lot of undiscussed issues about how to construct the framework.

For example, a key point in these analyzes is the perspective of the study? How will success be assessed? In the health sector, we tell the story of the “exploding cigarette” that is highly addictive, generates large tax revenues, but kills the person on the day of retirement, so the government does not incur later health care and pension costs. From a fiscal perspective, this is an ideal policy, except that it bypasses all considerations of individual well-being.

Before you say that ignoring people’s well-being is absurd, I hear that the government’s SIA is focused on fiscal savings for the government. So the perspective is that the rich benefit more than the poor. I hope this is not true, because in every health assessment I have been involved in, the suggested treatment involved net public expenditure in the long run because it was intended to benefit the sick.

I gave just one example of the minefields we found while compiling the WHO report. Everyone will be there at SIA and probably some of them. The health assessment projects I have been involved in often lacked quantitative evidence; sometimes this evidence is the key to the assessment.

The serious issue is, as the term ‘investment’ suggests, that there is a strong time element to the valuation. (No, I am not going to go back and forth on the discount rate over time – I think we got it wrong forty years ago and are still getting it wrong.) Rather, the analysis requires evidence about the impact of policy decades later. For example, good nutrition in pregnancy affects the health of their grandchildren. (The mechanism seems to be through an impact on the health of the baby in the womb, which in the case of a girl will affect her children.) Much of this evidence is not available for social care issues. (The health sector is much better equipped with the evidence required than social services, but even then it is often lacking.)

The temptation for an ombudsman – or a consultant advising an ombudsman – is to come up with the missing numbers. Sometimes policy proposals depend on their size; you won’t be surprised that there is an optimistic attitude in their creativity. This attitude will feed politicians.

CBA and SIA are just informed decisions. They are rarely decisive. The coalition government agreement said it would make decisions that were “principled – made on the basis of public policy principles, including problem definition, rigorous cost-benefit analysis and economic efficiency”. This was almost certainly written by someone who had never done CBA.

In practice, the government seems unaware of this principle. This is most visible in the area of justice, such three-month training camps. My impression – I’m not an expert on this – is that the evidence says they won’t work. Did the politician commission any SIA assessment – let alone a rigorous one – before ordering the policy to be implemented? (Will he order follow-up evaluations?) How many of the current government’s policy proposals have been delivered with the promised severity? It would be useful if every time a policy was announced, a journalist asked to see a “rigorous cost-benefit analysis.”

SIAs could make a significant contribution to public policy if they forced us to focus on evidence to inform it. (In the health sector, this is called “evidence-based medicine.”) I’m not optimistic; it involves a sea change in the way we make public policy.

*Brian Easton, independent scholar, economist, social statistician, public policy analyst and historian. He was Listener economics columnist from 1978 to 2014. This is a re-post article originally published on pundit.co.nz. He is here with permission.