The 1989 Public Finance Act distinguished ‘outputs’ from ‘outcomes’. Outputs are what a department (or, more generally, an agent) can deliver while outcomes are what the minister (or, more generally, the principal) actually wants. Thus a minister may want, on behalf of the country, a high level of education in the population, but a school can only promise to meet certain educational standards in its students, say in literacy, which – ideally – are related to the desired outcome but are not the same thing.

Critically, the output is the measure for which the agent is held to account. Thus Gilling’s Law, formulated by Don Gilling when he was professor of accounting and finance at the University of Waikato, states that the way you score the game shapes the way the game is played. An obvious instance is that in rugby giving tries higher points changed incentives so more of them were scored. Fortunately this output was well-aligned with the desired outcome – winning or, better still, winning an attractive game. But that is not always true; it can be very difficult to align outputs with outcomes.

The easy solution is to redefine the output as the outcome. A recent paper by Des Gorman and Murray Horn, both prominent in the management of the healthcare system, does exactly that. (‘Purchasing Better, Innovative and Integrated Services’, Internal Medicine Journal 45, 2015) It calls the outputs of the health system ‘outcomes’. But the outcome we want from the health system is not so many surgical operations – what it can deliver – but better health. The operations or whatever may contribute to better health, but they are not the same thing. One could imagine a less responsible hospital doing the easy operations to attain its targets, leaving the challenging ones aside even if their health outcomes were higher.

The same problems bedevil the education sector. For example, the stated objective of the Performance Based Research Fund at the tertiary level is ‘to ensure that excellent research in the tertiary education sector is encouraged and rewarded’. But the research it rewards may have little connection with the education with which an institution is concerned. It weights full-time researchers exactly the same as full-time teachers. Ironically so, as universities trumpet their PBRF scores to attract students. They do not mention that their students may rarely meet the high achievement researchers and that many of their teachers may not have been included in the PBRF exercise.

That reminds us that not only has PBRF distorted the purpose of university education – to provide an education by teachers who were on the cutting edge of research (see below for the statutory outcomes[i]) – but that the measurement system itself has been distorted. Some of the practices that have been cynically pursued by university administrators in the calculation of PBRF scores would result in student being sent down if they had used them when being assessed. Some examples of the problems are here, here and here.

The misalignment of or confusion with outputs and outcomes is not peculiar to tertiary education. Although not generally the way it was presented, the row over the NCEA standards was about the degree to which the outputs were aligned with the real educational outcomes with which the sector is concerned. The fear has been that the objectives of the system would distorted towards what was being scored.

It is instructive of how poorly these output measures can be thought through. A high average NCEA score is not even the output we want. Economists distinguish between ‘net output’ and ‘gross output’. There have been economic regimes whose incentives are to maximise gross output; they are often called ‘Stalinist’ after the approach of his times. It is an inefficient system in the sense that it in encourages wasteful production as the record of the Soviet economy well illustrates. Modern economies seek to maximise net output, or value added, that is the difference between output and inputs; they have a much better performance record compared to the Soviet-type ones.

In the case of schools the aim ought to be to improve the student’s attainment, so that it is a greater achievement to start off with a student from an educationally underprivileged background and give her or him a high score than it is to take a student from a privileged background and put in a little effort to result in the highest score.

Instead Stalinism reigns and, just as for the PBRF, the educational system is being distorted by the NCEA. Some schools have concluded that they can up their average score by excluding students who are from underprivileged backgrounds or are otherwise hard to teach. Their challenge is how to do this without it being too obvious. Apparently a few have been caught.

Thinking rigorously about the output-outcome distinction is likely to prove a challenge to the Productivity Commission which is charged with reviewing the tertiary sector. Productivity is measured by the ratio of outputs to inputs. In a conventional market the output of a business aligns with the required outcome, but it does not where the ‘market’ works differently, as occurs in education, health and social services.

The danger is that the Commission will lapse into traditional thinking and add to the distortions from which tertiary education and training already suffers from the misalignment of outputs and outcomes.

It is vital that those sectors which are not driven by the market in the conventional way challenge the established wisdom with its confusion between outputs and outcomes and its dominance by generic managers who are good at pursuing outputs but who have no sympathy for the sector outcomes.

Ideally the challenge to the conventional wisdom needs to be from an alliance of sectors which are suffering from the confusions of the current regime. However the immediate point of contestation is the Productivity Commission review. It would be heartening if the university sector was to rise to the challenge; it would involve diverting some of their very best thinkers to do so – with, no doubt, a reduction in their PBRF scores.

[i] There is a tendency to set outputs without even addressing the outcomes set out in statute. Here they are for universities. Observe especially the ‘primarily’ and ‘principal’ in the first subsection and ponder the extent to which the PBRF took them into consideration.

Under Section 162 of the Education Act (1989) a university is defined as having the following characteristics:

(i) They are primarily concerned with more advanced learning, the principal aim being to develop intellectual independence:

(ii) Their research and teaching are closely interdependent and most of their teaching is done by people who are active in advancing knowledge:

(iii) They meet international standards of research and teaching:

(iv) They are a repository of knowledge and expertise: and

(v) They accept a role as critic and conscience of society.