Designing KPIs that are Robust to Manipulation: KPIs, Salaries and Incentivised Action

Jeremy Chen

During GE2011, PM Lee pledged PAP will change way it governs Singapore

I would like to talk about developing KPIs that are robust to manipulation. Before beginning, I would like to apologize in advance that this article will be slightly technical. It has to be as KPIs have to be properly designed to be aligned with social objectives (such as “inclusive growth”).

This alignment with social objectives has to be done with the maximizing behaviour of individual policy makers in mind. It is thus natural to begin with a discussion on how systems of KPIs are “gamed” by those who are the subject of measurement, since their pay-offs (salaries) are essentially (increasing) functions of the KPIs.

Following the discussion on how KPIs are manipulated, I will cover why KPI manipulation should be taken into account. Then, I will comment on the manipulability properties of the current salary scheme. Finally, principles by which robust KPIs may be designed will be discussed. In the annex, I will run through the process of creating a salary scheme aligned with the objective of encouraging “inclusive growth”.

Preliminaries on KPI Manipulation

As a preliminary to what follows, it should be stated clearly that the process of manipulation of a system of KPIs is essentially one of performing optimization to maximize one’s pay-off. (Here, the term optimization is used here in the sense of attempting to maximize some pay-off function over all possible decisions.) In practice, optimization is based on manipulating the (policy) levers that one has control over.

Having spoken briefly about manipulation, let us consider “growth”. One might summarize all forms of economic and technological engineering as effort directed at two objectives that may be concurrent: (i) improving efficiency to reach the efficiency frontier (where it is possible for every aspect to be improved without making any aspect perform more poorly; “transitioning to a better indifference curve”), and (ii) making the trade-offs that maximize one’s benefit. Of the two, only (i) may be thought of as true growth and happens to be the more challenging objective, while (ii) is the task of selecting, from economically efficient alternatives, the one most beneficial to oneself.

Now, we are concerned about policy makers trading something that benefits them less for something that benefits them more. In other words, KPI manipulation. When a policy maker does that to increase his own pay-off, winners and losers are created. In simple economic (game theoretic) analysis (which assumes self-interested behaviour), the pay-offs for those winners and losers do not matter to the policy maker, who is only interested in his own pay-off.

Why Bother?

We cannot be so naive as to assume that intelligent people will not optimize for their own benefit. This will happen, and our policy makers are, at the very least, not stupid. As such, incentives have to be set up such that the maximizing behaviour of policy makers are aligned with social objectives.

To be cynical about the present government, a peg to the salaries top earners generates the incentive to implement precisely two types of policies: (i) those that encourage higher salaries for top earners and (ii) those which trade the welfare of non-top earners for higher salaries at the top. The only defence against this behaviour would be “human goodness” or “the threat of non-election”. Game theory and common sense would tell us that appeals to the former are nothing but cheap talk. This configuration of incentives is undesirable. Incentives should be in place to directly support “inclusive growth” (or whatever policy objective is articulated) rather than leaving the objective out of the incentive system (which makes sense to office holders only if it is there for public consumption and is not, in fact, the true objective).

The current and proposed ministerial salary systems are textbook examples of incentive systems which are not aligned with the stated policy objective (“inclusive growth”). Subsequently, I will do a cursory analysis of the current ministerial salary scheme, which I will abbreviate as MSS.

An Analysis of the Current Salary Scheme

One important observation that can be made about the MSS is that some aspects of the growth objective are weighed much more heavily than others. In the MSS case, the ratio of the weight given to the salaries of top earners to those of low-income workers is extreme. Without bonuses (which is based on GDP growth), this ratio is infinity (i.e.: the income of low-income workers do not matter at all). Thus, the underlying optimization problem might be framed (polemically) as such:

maximize Expected Salaries
over: Feasible Economic Policies
subject to: Probability[Re-election with Comfortable Majority] ≥ 1-δ

An over-simplified, but concrete, (and still deliberately polemical) version of this might look like:

maximize 0.6 Expected Median Salary of Top 1000 Earners (EMS)
over: non-negative EMS and Expected Median Salary of Singapore Citizens (EMSSC)
where: α EMS + β EMSSC ≤ γ (Efficiency frontier type Constraint)
subject to: EMSSC ≥ λ (“Re-election Constraint”)

which has optimal solution [EMS, EMSSC] = [(γ-βλ)/α, λ]. This solution informs us that (i) only the bare minimum will be done to “ensure re-election” and (ii) all other economic capacity will be directed towards boosting the salaries of top earners.

It should be recognized that this is a simple model designed to be polemical. However, it highlights the fact that when (intelligent) policy makers optimize for their own benefit, it can negatively affect general welfare unless their incentives are aligned with the general good.

Principles of Design for a KPI System/Salary Scheme

“Inclusive growth” is a policy objective of the “general welfare” type. The other being of the “narrow quantum leap” variety such as “putting an man on the moon by the end of the 1960s”. The former entails spreading out resources and the latter entails focusing them. Presently, I’d like to make suggestions of the kind of ingredients that might go into KPIs/salary formulas for policy objectives of the “general welfare” variety.

To ensure that no one is left behind, it is important to ensure that the KPIs do not make it profitable to lower the welfare of one group in favour of another. For instance, with a KPI such as C1 [Factor A] + C2 [Factor B], and a technological/economic aspect that allows trading of 1 unit of [Factor A] for 2 units of [Factor B] using 1 unit of “policy effort”, unless the coefficient C1 is close enough to 2 C2, trade offs will be made that lower one of the factors to its minimum level. (For this example, if C1 – 2C2 exceeds 1, [Factor A] will be increased and [Factor B] decreased; and if 2C2 – C1 exceeds 1, [Factor B] will be increased and [Factor A] decreased. Also, for the more mathematically inclined, this example clearly brings out how “linear” KPIs without feasibility conditions/veto criteria can be dangerous.)

Trade offs are the essential mechanism by which KPIs are manipulated. (I know of no others and would be keen to learn of others.) If the effort to make a trade off is worthwhile for the self-interested policy maker, the motivation to make that trade-off will exist. This principle has other serious implications in the public sector which I decline to touch on at this point.

Now, the converse to the aforementioned principle (which is also true), is that if the effort to make a trade off is not worthwhile for the self-interested policy maker, the motivation to make that trade-off will not exist. I believe that if growing the pie becomes the only practical means of increasing their rewards and remuneration, policy makers’ efforts can be focused to that end and it will be possible to promote objectives like “inclusive growth”.

As a rule of thumb, the more broad-based a KPI is, the more effort required to make gains due to trade-offs. Furthermore, a broad-based KPI is precisely what is needed to measure “inclusive growth”. (The more mathematically inclined might think of using a function of the minimum of a set of subsidiary KPIs, or a function of the a set of the “order statistics” of subsidiary KPIs, which would require that all/most sub-KPIs rise in order for the parent KPI to rise.)

Conclusion

I hope the forgoing discussion was useful and helped stimulate thought on how to design KPIs or a salary scheme to achieve policy objectives. We have discussed how the ability to make trade-offs allows policy makers to optimize their KPIs without growing the economic pie (or the per-capita economic pie) and how this can be dangerous. It has been suggested that if these trade-offs are no longer easy to make, it would be more profitable for policy makers to work on “increasing the general welfare” (as a chief means for increasing their pay-offs).

I would like to close with a suggestion of a Ministerial Salary Scheme that I believe is more compatible with “inclusive growth” than the existing and proposed ones. I will describe the development of that scheme and what motivates the various components of it. I hope that the principles outlined in this article will eventually be used in the development of an improved (and more rigorous) salary scheme. (The reader should be warned that math will be encountered.)


Annex: A Proposed Ministerial Salary Scheme Designed for Inclusive-Growth

Note that this does not consider MP allowances, which are (rightly) paid over and above ministerial salaries. The salary scheme will be based on a benchmark salary paid to a junior minister, with senior grades getting (arbitrary) multiples of that salary. This benchmark will move in response to changes in economic conditions and the economic performance of Singapore. Let us denote the benchmark salary for the year Y, as SY.

Suppose a target salary is computed using a scheme like that suggested by former NMP Siew Kum Hong and call it C2012 (where the 2012 refers to the year of the original benchmark and “C” is for consumption). For those who have not read Siew Kum Hong opinion piece, S2012 is the price of a basket of goods and services that amounts to a decent standard of living for someone of a minister’s social standing. Now let CY be cost of the same basket of goods in year Y.

I would suggest that a minister’s salary be partially inflation adjusted to make it robust to market changes. (This would be one perk of office, and is entirely arbitrary.) Suppose that (arbitrarily) that amounts to one third of the original basket of goods. The rest should move with the wage levels of Singaporeans.

Define the income at the α fractile (highest income among the lowest earning 100α % of working Singapore citizens.) in year Y is Iα,Y, and determine Kα such that 2/3 C2012 = Kα Iα,Y.

So we find that S2012 = C2012 = (1/3) C2012 + Kα Iα,2012 for all values of α, and we can benchmark ministerial salaries as

SY = (1/3) CY + Kα Iα,Y

for any value of α. However, this pegs salaries to a particular income group, which introduces the motivation to manipulate public policy to increase the incomes of that group. Thus, we should broaden the base.

Suppose α were drawn from the set A = {0.05, 0.06, 0.07, …, 0.99}. I start from the 5% level to avoid pathological low-income cases (such as refusal to work) and end at the 99% level. Now for any set of positive weights w0.05, w0.06, w0.07, …, w0.99 corresponding to the elements of the set A such that they sum to 1 (Sum[α in A] wα = 1), the following holds: C2012 = (1/3) C2012 + Sum[α in A] wα Kα Iα,2012. This leads to a fairly broad based benchmark:

SY = (1/3) CY + Sum[α in A] wα Kα Iα,Y.

The weights wα should be reasonably even, perhaps even equal. However, I am of the mind that income inequality should be reduced, so it might make sense for there to be a small variation in weights for instance such that wα decreases with α and w0.05 = 2 w0.99.

Unemployment can be incorporated in this framework by including unemployed people in the income distribution. However, care should be taken to not build in the incentive for policy makers to introduce policies that introduce disincentives for home making and other economically valuable but unpaid work. This would require a lot more work to flesh out, so this will be left as an idea.

The final modification might appear a little complicated, but the idea is simple. We would like broad income growth and not income growth focused on the “easiest” part of the income distribution. Thus, we should consider the fractiles which have had the lowest growth since the benchmark year. Since A has 95 elements (95 possible values of α), we could perhaps consider 60 elements every year.

Let G(Y) be the set {Iα,Y/Iα,2012 : α in A} and let B(Y) be the set {α in A : Iα,Y/Iα,2012 is one of the bottom 60 elements of G(Y)}. Let the normalizing constant for the weights used, MY := 1 / (Sum[α in B(Y)] wα). (This ensures that MY Sum[α in B(Y)] wα = 1.)

Now, we arrive at the salary benchmark:

SY = (1/3) CY + MY Sum[α in B(Y)] wα Kα Iα,Y.

To arrive here, we have done the following:

  1. Determine a benchmark standard of living, an associated basket of goods and services and its price in the benchmark year.
  2. Split the benchmark salary into an inflation adjusted component (perk!) and a market adjusted component which depends on the incomes of Singaporean workers.
  3. Made the benchmark depend on multiple income fractile points to account for the standards of living of a broad range of Singaporeans.
  4. Set the weights associated with each salary benchmark to promote the reduction of income inequality, while ensuring that the weights are not “badly skewed”. (Note: In effect, this is a weighted average of all the salary benchmarks of item 2.)
  5. Decided to use only a sub-set of the elements of A to compute the weighted average. The elements used relate to the income levels which have grown the least since the benchmark year. This promotes the raising of all income levels, and punishes ministers (with stagnating wages) for stagnating wages of their constituents.

Consider the following examples of how the “performance” term (the second term) varies with changes in income. (i) all incomes rise {fall} by 1%, the “performance” term rises {falls} by 1%; (ii) income levels at 60 of the fractiles in the set A remain the same and the rest rise, the “performance” term remains constant (since the worst 60 fractiles are used). These examples illustrate the idea of inclusive growth.

I hope this portion has been informative and interesting. I must emphasize that this is just an example of how to apply the ideas in the forgoing article. However, I hope that such ideas might be used to develop a salary benchmark in a more rigorous fashion.

—-

Afternote 1: A final element of this scheme would be conditions for when a re-benchmarking can be called for. My sense is that it would be sensible for this to be done after each election. Also, I feel that the President’s pay should be entirely inflation adjusted.

Afternote 2: It does strike me that a 1% rise in part of pay for a 1% overall rise in income seems stingy. However, it is arguable that while government policies are able to torpedo incomes, the dominant cause of rising incomes are drive and intelligent action on the part of individuals. To postulate a further bonus for income growth does not make much sense.

The problem with this is that indifference to facilitating income growth might be encouraged on the part of office holders. It this makes sense is to add loss aversion into the mix (c.f.: prospect theory, which led to a Nobel Prize in Economics), where drops in income at the reference fractiles are penalized more heavily than increases are rewarded.

Afternote 3: It is entirely possible to use a similar mechanism to allow salaries to rise to match that of top earners if national income growth targets are met. (Naturally, I refer to stretch goals.) The functional form of such a benchmark, RY (“R” for reward), might look like the following:

RY = SY + (ITarget,Y – SY) f(GY, GTarget )

where ITarget,Y is some target “top rate” income for the year Y,

GY = {MY Sum[α in B(Y)] wα Kα (Iα,Y / Iα,2012) } – 1

is a measure of “inclusive growth”, GTarget is its associated (stretch) target and f is a non-negative function that is increasing in the first parameter and decreasing in the second. (e.g.: f(x, y) = Max(0, x/y)2 or f(x, y) = Min(Max(0, x/y)2), Max(0, 2x/y – 1)) which is more reasonable.)

Afternote 4: It is appears that the idea of designing KPIs to make effortful manipulation non-profitable has not been mentioned in the academic literature. I would like to pre-emptively coin the term “friction” for it.

Photo courtesy of Bloomberg