Australian Health Promotion Association

Ben J. Smith
Editor-in-Chief, Health Promotion Journal of Australia and
Department of Health Social Science, Monash University

The opportunity I have had in recent years to facilitate a number of training courses with health promotion practitioners, has affirmed the value that many place upon evaluation in program management, and provided insights into the perceived benefits, costs and risks that this may entail.

As might be expected, course participants can readily identify the benefits to be gained from evaluation: finding out what works; improving how programs are delivered; adding to the evidence that others can use; increasing the prospects of future funding; and satisfaction and rewards for workers, among an array of others.

When attention turns to the other side the ledger – the costs and risks of evaluation – the discussion heats up. Among these are the time and financial costs of evaluation, and the drain this puts on the limited resources for program delivery. There is the risk of discovering that the program was not effective, and potentially losing funding (and even your job). Misrepresenting the nature and effects of programs is another concern, because of difficulties in measuring impacts, limited evaluation skills, and the challenge of undertaking systematic evaluation in the fluid environment in which strategies are conducted. Not long into these discussions, the list of costs and risks is as long as that of the benefits, and it is necessary to further reflect why evaluation is, after all, a worthwhile endeavour.

These insights may go some way towards explaining the discrepancy between the espoused value given to evaluation in health promotion and the extent to which this is put into practice. In Australia, evaluation skills are identified as core competencies for health promotion and are rated as important by practitioners.1 Yet, it has also recognised that many health promotion programs are not evaluated, or at least not in a way that generates evidence to inform practice.2-3

So, are we making progress in the evaluation of health promotion programs in Australia? The answer would appear to be: slowly, at best. While there are numerous examples of programs that have excellent evaluations at the formative, process, impact and outcome levels, there are many others that represent missed opportunities for learning. These are missed opportunities for practitioners, the agencies in which they work, funding bodies and, of course, the population groups that may not benefit from the improvement and ongoing implementation of health promotion strategies. The need to improve the capacity for, and quality of, evaluation in health promotion has also been highlighted in the United States4,5 and Canada.6

There is some evidence that workforce development initiatives can improve evaluation knowledge, confidence and practice.  These may include training courses, guidelines and tools, access to expert technical assistance, and mentoring of staff.7,8 However, our understanding and experience of capacity building in health promotion, which is supported by consultation with practitioners,9 is that organisational and systemic changes are required to enable more extensive and higher quality evaluation.

Before offering some thoughts on the kinds of organisational and systemic changes that may strengthen evaluation in health promotion in Australia, I would suggest that this is a topic that deserves greater attention and analysis across our field. An outcome of this may be improved planning or indeed advocacy to achieve this much needed improvement in capacity.

Some key factors that deserve consideration include: clarity of performance expectations related to evaluation; adequate funding for evaluation; sufficient duration of funding for programs so that impact evaluation is feasible; allocation of time for evaluation within health promotion roles; and funding for specialist research and evaluation positions. A structural factor that appears to impede the evaluation work of many practitioners is that they work in isolation or in small teams. This reduces the pool of skills, resources and time available to devote to this work. The location of practitioners in larger teams may counter this effect. Facilitating linkages between practitioners and research and evaluation specialists based in universities and non-government organisations could also improve the technical proficiency and confidence for evaluation among practitioners. Leadership and funding is required to enable these partnerships to be established where they are most needed, over a sustained period.

The significant investment of government funds under the National Partnerships Agreement for Preventive Health10 is purported to be ushering in an expansion of health promotion initiatives in schools, child care centres, workplaces and local government areas around Australia. Strategic and adequately funded evaluation will contribute to making this an  pportunity for improving health promotion practice across diverse settings and population groups. It will also enable assessment of whether this investment has an impact on the determinants of population health, and warrants maintenance and increase over time.

Health Promotion Journal of Australia 2011; 22: (3)