Relationships – evaluation of a prisoner peer support programme

The translation of evaluation methods into practice very often hinges on one key element – the human factor. That is, the success or otherwise depends to a great extent on the human resources available, the quality of the individual contributions to the evaluation, and the level of conflict or cooperation associated with a particular project. Who does what and how can have major implications, not only for the evaluation process but also for longer-term relationships between evaluators and criminal justice officials. This is demonstrated in our second example, which, as summarized in Box 22. 3, consisted of an evaluation of a prisoner peer support programme.

A core function of this evaluation was to inform ongoing improvement of the Red Cross peer support programme and its delivery in the Tasmania Prison Service. The evaluation process focused on the way in which the programme approached and met

stated outcomes, by examining whether peer support structures and mentoring activities promoted positive change in inmate choices and behaviour. The evaluation was intended to strengthen evidence-based practice and to generate knowledge to inform future funding submissions, social policy positions and advocacy initiatives within Tasmania and more widely across the Australian Red Cross.

A distinguishing characteristic of impact evaluation is the emphasis and priority given to establishing the effect of the initiative (in this case, the Red Cross peer sup- port programme). In addition to considering traditional evaluation criteria such as the context or environment within which the programme is operating, the evaluation examined the effects (positive and negative, intended and unintended) of the pro- gramme. Six key questions guided this evaluation:

1. What is the impact of the peer support programme on peer supporters, prisoners, correctional officers and ancillary care service providers working within the prison system?

2. Is the peer support programme having a demonstrable effect?

3. What is the perceived quality of the programme?

4. Is the programme operating as intended?

5. What are the comparative cost benefits of the programme?

6. How relevant are the supporting training manuals, measurement frameworks and policy/procedure materials?

The impact of this programme was measured in a number of different ways and data collected from varying sources. Key documents relating to the programme, collected for analysis from the Australian Red Cross and the Tasmania Prison Service, included such things as prison guidelines and policies, the Red Cross prisoner support pro- gramme operations manual, peer support programme implementation policies, peer supporter data collection sheets, and so on. These supporting documents were evaluated in the context of practical activities and shared understandings of purpose. Interviews were conducted with 23 key stakeholders involved in or associated with the programme in some way. All participants, with the exception of the Red Cross project officer, were recruited with the assistance of the Tasmania Prison Service. Peer supporters and inmates using the service were recruited with the assis- tance of the Red Cross project officer, who introduced peer supporters to the UTAS evaluation team. The evaluation objectives were explained to potential participants in a joint presentation by the Red Cross project officer and the UTAS evaluator. Following expressions of interest from peer supporters and from other prisoners, separate appointments were made to conduct interviews with participants. Those interviewed included five peer supporters, seven inmates, eight correctional staff, two integrated offender management staff and the Red Cross project officer. A num- ber of ‘impromptu’ conversations also occurred, particularly with correctional officers, in the course of the evaluator’s visits to the prison. Although they declined to be formally interviewed, these officers were more than willing to provide input

and opinions about the peer support programme ‘off the record’. Their views are not recorded or presented in the final evaluation report; however, they did influence the interpretation of the documents and interviews formally relied on for the evaluation. A limitation of the evaluation and the programme itself was that, at times, data collection was interrupted by the uncertainty and unpredictability of the prison envi- ronment, including lock-downs at short notice. Data collection took place over a period of five months between August and December 2010, and then over a further five-month period between March and August 2011. During these periods, there were occasions when appointments were made, postponed and cancelled due to confusion among correctional officers about the purpose of the visits, due to lock- downs, and due to timetable logistics within the prison. There were in fact a number of different issues associated with the interviewing process. The evaluators and oth- ers involved in the evaluation were variously frustrated in their efforts at different

times and for diverse reasons.

Overall, the evaluation found that the tenets of the programme are consistent with the Australian Red Cross (ARC) commitment to caring for those on the margins of Australian society; supporting young people; improving opportunities, resilience and positive social connections; and assisting inmates to develop mature coping skills surrounding relationship and family issues, thereby helping families to better cope with having their family member incarcerated. The key evaluation finding was that, by and large, the peer support programme was working well and effectively in what can be, at times, a volatile and chaotic work and living environment. Key strengths of the programme were summarized and a number of areas where specific action could be taken to improve existing performance were recommended.

The issue that arose during the course of this evaluation was one pertaining to damage or potential damage to existing and future relationships – between the evaluation team and the prison, and between the Australian Red Cross and the prison. This was due primarily to the hiring of an interviewer for the evaluation team who, unknown to myself, was actively hostile to prison and those who work within such institutions. Her ‘relationship’ with those she interviewed, especially among correctional officers, was basically poisonous. This not only skewed the interviews in particularly negative directions, but it also meant that pursuing interviews was made that much more difficult as the evaluation proceeded. One consequence of this was that the evaluation was briefly suspended, and new interviewers, including myself, were thrust into the assessment process. This enabled a friendlier atmosphere to be constructed and reclaimed. It also provided for a number of additional interviews to be carried out – at the evaluator’s expense in terms of additional time and money – so that a valid and insightful evaluation could be provided.

One lesson from this is that the ethics of evaluation is about ‘trust’, ‘honesty’ and

‘reliability’. It means treating people fairly and openly. It implies a commitment to the physical, psychological and emotional well-being of everyone concerned. Ethics is not simply about ‘correct procedures’ – it is about human relationships. In under- taking criminological evaluation, the evaluators must therefore be conscious of any

conflict of interest involving members of the evaluation team, including any espe- cially ideological partiality that impacts on the evaluation process. Knowledge of codes of conduct for members of an evaluation team in relation to participants is important, as is recognition of the responsibilities of evaluators in regards to proce- dural rules and protocols.

Fortunately, the deficiencies in data collection and quality of data were detected before too much damage was done. Moreover, the funding body (the Australian Red Cross) was acutely aware of the need not only for robust data, but also for positive ongoing relationships at the prison. Extra work put in by the evaluation team, including volunteer labour, ensured that the final evaluation report was of a high standard and was a valuable source of information for the commissioning body. Strong relationships of trust (and forbearance) were also maintained across various communities of interest.