Step 9: Utilisation

Evaluations are only worthwhile if they are used. For this reason, systematically preparation and encouraging the utilisation of the evaluation is just as much a part of the ninth step of the evaluation process as the actual utilisation of the evaluation itself.

Bauer mit Apfelkiste

For the longest time, the utilisation of an evaluation was understood to mean exclusively the use of evaluation findings, i.e. everything presented in the (final) evaluation report. It is now widely recognised that the evaluation process itself can already bring a wide range of benefits to the object of evaluation. In other words, participation in the evaluation process itself - whether as a client, manager or data provider - can lead to (desired) changes in individual thought and action or to reflexive learning. This happens when one deals intensively with evaluation goals and also undesirable findings regarding the object of evaluation and when one is exposed to the expectations of other stakeholders and possibly interacts directly with these stakeholders. These examples of utilisation can all have an impact on the organisational units responsible for the object of evaluation or on its political or social context - even while the evaluation is still in progress.

There are several types of utilisation: Instrumental utilisation means the use of the evaluation and its findings by relevant stakeholders, in particular for improvement and fundamental decisions. Conceptual utilisation is understood to mean including or processing evaluation findings in research findings by the scientific community, for example, or in suggestions to members of a given profession or in advocacy work to influence public opinion.

Rather undesirable cases of utilisation occur when evaluation findings are used to justify decisions already reached and not disclosed before the evaluation was carried out or when the evaluation is only carried out for show.

The utilisation of the evaluation and its findings varies in intensity and is handled differently by different addressees. Important factors that encourage utilisation include:

  • When it is decided in advance during the evaluation planning phase how evaluation findings will be disseminated and utilised;
  • When the relevant stakeholders are involved at an early stage of the evaluation planning process, making stakeholders more willing to utilise evaluation findings;
  • When it is clear to everyone how evaluation findings will be utilised, and when a relationship of trust is built between evaluators and stakeholders;
  • When evaluation findings are prepared and submitted in a prompt and timely manner;
  • When the evaluation is carefully prepared, especially in cases where the focus of the evaluation is on accountability or a fundamental decision. This also ensures that any ‘undesirable’ evaluation findings are nevertheless utilised.

Desired utilisation can become a challenge when there is fluctuation of important stakeholders. Substitution arrangements can help to counteract this problem.

There are also improper forms of utilisation. This includes misuse of evaluation findings, unintentional occurrences or intentional non-utilisation. Such improper forms can affect evaluators and stakeholders alike, especially in evaluations where there are high risks for stakeholders, e.g. when fundamental decisions have to be made. In order to counteract these risks, it is worth deciding at an early stage how evaluation findings will be handled and how decisions will be reached. It should also be clearly established what constitutes professionally and ethically unacceptable utilisation if misuse nevertheless occurs.

Finally, utilisation should be explicitly documented by maintaining a record of which reports were sent to whom, where these reports were discussed, what discussions on the evaluation findings took place and what consequences resulted from them.

Evaluations may (1) have an intended or unintended impact; (2) this impact may be based on evaluation findings or on the evaluation process, and (3) this impact may occur during or directly after the evaluation, and even long after the evaluation is completed.

It is not enough to merely present and convey information. Evaluators must also consider how to ensure that this information will be utilised. This requires follow-up activities, e.g. information campaigns for consumers. Finally, evaluation results must have a tangible impact. For example, traders may reduce the selling price or remove the apple from their product range. Or apple farmers may make changes to the way they cultivate apples and then look at the new variety of apples with a sense of satisfaction, as shown in the illustration.