Insights

In designing grant awards and developing workplans with local organizations, we often place a lot of emphasis on setting targets. Theoretically, targets should help us focus on achieving quality results and to make it easier to hold ourselves and our partners accountable for performance. This should lead to better results overall and lasting impact. But does it?

I was recently thinking about how we encourage our grantees to take adaptive management approaches and adjust their strategies in the face of evidence. In view of the complexity of development issues and an increasingly unpredictable world with pandemics and climate events, adaptive management approaches are critical. But the use of, or perhaps the misuse of, targets can undermine this agenda. Once targets are set, implementing organizations tend to pursue them obsessively and ignore signs that the targets need to be changed or that the wrong targets were set. In program implementation, there is a strong bias towards following rather than questioning the initial plan. Staying the course we set for ourselves is the easiest option. Questioning the plan requires analyzing data, asking uncomfortable questions, deciding course correction is needed, and most importantly, coming up with a better plan. When you base funding agreements on targets as NPI EXPAND often does, having to revise targets also requires an additional administrative burden to modify grants.

The first problem with targets is that they are often set for indicators that are the easiest to count, but not the most critical for the higher, sustainable goals we are trying to achieve. Take the example of contraceptive social marketing. Two of NPI EXPAND’s local partners, Jigi in Mali and ADEMAS in Senegal, are product social marketing organizations. As is generally the practice, they are tracking and reporting product sales as one of their main targets. Product sales are easy to count because we have a sales record that shows when the products were transferred from the social marketing organization to the wholesalers who eventually get them to consumers. But what are sales really measuring? We don’t know when the products are actually used by consumers, who those consumers are, or if they are using the products correctly. A good family planning program should focus on reaching underserved consumers and attracting new users. But if every sale counts the same toward your target, a program manager is likely to focus efforts where sales are easiest—with current users in urban areas. Making the extra effort to help new users to choose the right family planning method or to conduct more targeted distribution in underserved areas would be more in line with the goals of a successful family planning program, but it would require more time and more funding to achieve fewer sales. NPI EXPAND utilizes fixed amount awards with these organizations, where we pay for outputs, but sales are not among the outputs we pay for. Instead, we pay for marketing and distribution plans that are evidence-based and data collection systems that allow us to see how well our grantees are reaching their target consumers. Sometimes, the right balance can be achieved by adding a quality “modifier” to a target. So, for example, in addition to a target for sales, one could add a more heavily weighted target for sales to new users and for sales in underserved areas. Introducing these quality elements usually requires additional funding to put the monitoring systems in place to capture these results. In other cases, if we are funding training programs, we include references to established standards or existing curricula approved by an appropriate authority such as the Ministry of Health or the World Health Organization.

A number of our awardees also worked on COVID mitigation programs and supported risk prevention communications. A typical indicator that is used for communications activities is “numbers of people reached with prevention messages.” By reached, we typically mean the people heard or saw the message that was disseminated via mass media channels or interpersonal communications. This is relatively easy to count, but the number of people reached doesn’t tell us anything about the numbers of people who found the message convincing, and even less about the number of people who adopted COVID prevention behaviors. These are the results that we are most interested in and which show the most impact of programming, but measuring those results requires complex surveys of the target audience.

The other big issue with targets is that they are easy to “game” either in terms of finding shortcuts to reach the targets or setting them at such a low level that it becomes easy to say that all targets were met or exceeded. When implementers set targets, they tend to exaggerate the difficulty of reaching targets to be sure of being able to meet or exceed the targets. This can be managed through negotiation between the grantor and grantee, but the pressure to demonstrate success usually leads to modest targets. Of course, some grantees go in the opposite direction and want to impress their donors by proposing heroic targets. Unfortunately, when they fall short they look like they are failing when they might be doing a good job. Both of these situations stem from the often-arbitrary nature of target setting.

So what is the solution? Should we do away with targets entirely? Just because targets are not as meaningful as we are sometimes led to believe, it doesn’t mean they are meaningless. Often, there are no good substitutes for proxy indicators that can be captured with the time and resources available. Even the most fanatic evaluation expert wouldn’t advocate for using randomized control trials to assess every development activity. The cost of evaluating the programs would far exceed the programs themselves. Here are a few recommendations for using targets wisely:

• Know the limitations of your target indicators. If it is a proxy indicator (such as using couple years of protection as an indicator for contraception use), know that it is just a proxy and verify that the trends in the proxy indicators align with the trends of the main indicator.
• Don’t impose targets on program implementers. Donors and grant makers often require targets in key result areas as a way of pushing program implementers to focus on what they believe is most important. But with some global health programs that are working toward global targets, there is a risk that targets are imposed on grantees such that grantees focus exclusively on the targets to the detriment of more sustainable, effective programming. Realistic targets should be negotiated rather than imposed on implementing organizations to ensure that the organizations trying to reach them understand and own them.
• Design a balanced monitoring, evaluation, and learning plan. Short term indicators may provide one part of the picture, but a fuller picture of program effectiveness has to come from more rigorous evaluation methods and qualitative assessments. This might involve launching new research for your program, but it could also involve leveraging other data collection and analysis processes that exist which can provide that broader perspective at no additional cost to your program.
• Build in mechanisms to ensure adaptive management. To overcome the bias towards staying the course at all costs, programs should conduct regular performance reviews that go beyond just assessing what percentage of our targets have been reached and ask the harder questions about whether we are making progress in achieving our higher, long-term objectives. Involve donors and stakeholders in those reviews so if program strategy adjustments prove to be needed, these stakeholders will support necessary program changes.
• Communicate early and often about what is most important. Program managers have to emphasize that it is the end result that implementers should keep in mind. Staff should be incentivized to maintain quality standards in implementation, and the higher order goals, not just quantifiable outputs. This increases the chances that our short-term process outputs will bring about our desired outcomes. Communicate this through financial incentives as well—don’t link all funding to quantitative targets; use targets with quality modifiers that balance easy to measure results with higher objectives.

Jeff Barnes, NPI EXPAND Project Director