 |
STORIES FROM THE FIELD. Politics – evaluation of the ‘Inside Out’ Prison Program
|
|
|
|
STORIES FROM THE FIELD
The remainder of the chapter provides sketches of three evaluation projects, each of which involved different objectives and methods of assessment, and most instructive, very specific problems and dilemmas arising from each evaluation.
Politics – evaluation of the ‘Inside Out’ Prison Program
Evaluation is frequently beset by ‘politics’, and in particular the manipulation of processes and outcomes to achieve certain (sometimes preordained) ends. This can create problems for criminological evaluation that strives to be fair, objective and unbiased. No academic research is ever ‘value-free’, but good research, including evaluation, demands that the high standards of rigour, validity, transparency and ethical conduct be applied (Australasian Evaluation Society, 1998). This is illustrated in the discussion of an evaluation of the Inside Out Prison Program, the features of which are described in Box 22. 2.
 |
This evaluation was a summative evaluation, that is, one that is basically con- cerned with results. It takes place after a project or strategy has been in place for some time. The purpose of such evaluation is to monitor performance and outcomes up to a certain point in time. In this sense, it is retrospective (or backward-looking). It is concerned with learning from experience, in order to determine the impact of an initiative.
Two primary questions were central to the evaluation. These were: what are the benefits and strengths of the programme and what are the limitations and shortcom- ings of the programme? Data collected for the evaluation was derived from two main sources: documentation held by various agencies, such as the Attorney-General’s Department, the Department of Justice (Corporate Office), Corrective Services, Care and Communication Concern (CCC) and the Inside Out service; and interviews with major stakeholders, such as prisoners, prison officers, prison managers, support ser- vice providers and prisoner family members.
The key findings of the Inside Out Program evaluation were that:
· Prisoners valued the Inside Out Program and thought it provided a useful non- institutional avenue for inmates to reduce/relieve the stresses associated with incarceration by providing someone to talk to and communicate with, and that it enabled them to better liaise with their families and loved ones. It also provided an avenue to release pent-up emotions.
· Prison authorities have mixed attitudes towards the programme, due to (a) prob-
lems with how it was introduced and implemented; (b) how funding was estab- lished and subsequently allocated for the programme; (c) differing perceptions among prison officials regarding the status and role of a ‘lay’ worker within the prison environment; and (d) the perceived lack of reporting and accountability of a prison-funded programme.
· From a service provision perspective, while the Inside Out support worker was
|
|
|
highly committed and motivated to assist prisoners and their families, there were serious shortcomings in the support available to the worker – both from prison authorities (such as provision of adequate space, or suitable report back/consultative mechanisms) and from the funded organization (in the form of a locally based support network and constructive administrative and supervisory support).
· The support worker himself was inevitably placed in an ambiguous position due
to the nature of the work. On the one hand, to do the job effectively requires certain personal qualities and task-related attributes. On the other hand, these very qualities and attributes may be seen as inappropriate in secure punishment facilities. This is a Catch 22 situation that can only be resolved by careful consid- eration of the philosophical rationale guiding prison management and prison programmes.
A number of recommendations were made about improving communication and consultation, and basically it was found that while the programme provided an
invaluable service to prisoners, there were aspects of the programme, particularly relating to reporting mechanisms and procedures, that required attention. Overall, however, the evaluation report stated that the programme represented excellent value for money, given the time, energy and resources put into direct service provi- sion by the key support worker, and the positive response from prisoners and their families to the service.
To understand the politics of criminological evaluation, several things need to be appreciated from the outset. First, evaluation research tends to revolve around spe- cific projects (e. g. a youth project that involves teaching young offenders about car maintenance and safety issues), specific programmes (e. g. a school programme designed to reduce truancy among students), or specific intervention strategies (e. g. use of juvenile conferencing as an alternative to court or to detention).
Second, evaluation research does not usually originate with the researcher. In most cases, evaluation will be commissioned by a state agency (such as an Education Department, Department of Justice, Health Department), a business organization, or a professional body, rather than being initiated by the researcher. This means that the terms of reference – what is being evaluated and how – are frequently predetermined by those funding the evaluation. This can influence the evaluation process in terms of budgets, who is consulted and the potential outcomes of the evaluation.
Third, evaluation research is ultimately about making judgements, and these, in turn, may have significant consequences for the actors involved in the evaluation, and such research also tends to be more overtly politicized than other types of social research. Values and interests are thus always close to the surface in evaluation research. The institutional environment within which agencies or businesses operate will shape how they construct and how they respond to the evaluation. In the light of this, it is vital that evaluators be vigilant in protecting the integrity of their work, and that they be clear about where they stand in the evaluation process (i. e. as inde- pendent evaluators rather than employees). From an evaluator’s perspective, the success or otherwise of any particular strategy or specific project cannot be taken for granted, no matter how good the intentions of the people involved or how laudable the goals and objectives.
|
|
|
In respect to this particular evaluation, two ‘political’ aspects stood out. On the one hand, from the very beginning the prison director had been critical of the way in which the programme had been foisted upon the prison by the Attorney General at the time. The Attorney General had unilaterally made the decision to introduce the programme, yet it was paid for out of the prison budget. Thus, there was internal antagonism to this particular programme due to the way in which it had been introduced.
On the other hand, the next Attorney General decided to axe the programme, in part due to overall government budget constraints. In doing so, she explicitly referred to the evaluation of the programme that we had carried out – and which did not recommend discontinuation of the programme but only that certain things could be done to enhance a programme that was deemed to be overall beneficial. Fortunately, after previously contacting the Secretary of Justice for permission, we
had published the evaluation on the Criminology Research Unit website and printed hard copies for general distribution. Thus, we were able to contradict the Minister’s pronouncement that the axing of the programme stemmed from our evaluation. By having publicly available documents, this ensured that, while the programme suf- fered, our reputation for integrity and independence did not. We did not provide the political justification for its demise and we were able to demonstrate this.
|
|
|
