 |
Thinking about doing evaluation
|
|
|
|
THINKING ABOUT DOING EVALUATION
Criminological evaluation can involve assessing:
· inputs (basic resources available to the programme, including staff, facilities, exist- ing networks, and so on)
· process (the number and nature of meetings, contacts and collaborations)
· outputs (which indicate numbers, of clients, for instance)
· outcomes (which indicate meaningful changes in clients, programmes).
Both quantitative and qualitative information are necessary to gauge the effective- ness, efficiency and degree of success of whatever it is that is being assessed (Australian Youth Foundation and Sharp, 1996; Murray et al., 1993; Wadsworth, 1991; White, 2013).
Evaluation involves research that speaks to the applied dimensions of criminology. That is, evaluation seeks to learn about the concrete effects and practical conse- quences of policies, programmes and projects carried out by government and non-government agencies and organizations. Evaluation projects can be carried out at different times; some are conducted alongside an intervention or an initiative; some are carried out at the end of a defined programme or project; while others involve evaluation on a periodic but continuous basis (for example, once a year or once every three years). When and how evaluation is undertaken reflects the pur- poses of the evaluation, the resources available and for whom it is being carried out.
Performance indicators
An important step in criminological evaluation is to establish the evaluation criteria or performance indicators (see White, 2013). These are the measures used to evalu- ate whether something is ‘successful’ or ‘unsuccessful’, ‘valuable’ or ‘not valuable’, ‘working’ or ‘not working’. Performance indicators refer to concrete ways in which performance is measured, whether in regard to processes or outcomes. They are signs by which we can determine whether a project is doing what its objectives and goals say it is doing. Performance indicators are generally ‘descriptive’ and ‘quantita- tive’ in nature. That is, they refer to actual developments, such as the holding of a meeting to discuss community safety, and quantifiable units of measurement, such as the number of meetings or of participants. Performance indicators indicate present
performance in relation to stated goals, such as how far along we are in achieving what we set out to achieve. They should not be confused with performance targets that refer to the end-point goals of a particular programme or project.
Performance indicators can serve different purposes, and reflect the agendas of those involved in the evaluation. They may reflect the immediate concerns of com- munity residents or service-users, who might be interested in how a programme or service is meeting their perceived needs. Alternatively, they may reflect the adminis- trative concerns of government and community agencies, which might be interested in whether or not their programme or service is reaching who it is intended to, or whether it is cost-effective. It is important, therefore, when designing performance indicators that general agreement is reached with the body commissioning the evalu- ation on both the desired goals and outcomes, and how these will be expressed in objective terms. The emphasis is that the meaning and measurement criteria of per- formance indicators are clear and agreed upon before the evaluation begins.
|
|
|
The development of performance indicators always takes place in the context of the evaluation’s objectives. It is essential to identity the specific need or goal (e. g. to reduce rates of recidivism among prisoners by providing intensive transitional sup- port); the particular meaning of the need or goal (e. g. what is meant by ‘reduce’, ‘recidivism’ ‘prisoners’, ‘intensive transitional support’); and the appropriate way to measure whether or not progress is being made relative to the goal (e. g. the number of prisoners recruited to the programme, recidivism rates prior to engagement in the programme, instances of re-offending post-release). As discussed in Box 22. 1, perfor- mance indicators need to refer to definitions and performance measures suited to the particular goals of a specific project.
 |
The choice of, and emphasis placed on, different performance indicators is partly a matter of what the evaluation is intended to do. That is, the aims of evaluation will determine the nature of the performance measures to be used.
|
|
|
