 |
Consequences – evaluation of the Post Release Options Program
|
|
|
|
Consequences – evaluation of the Post Release Options Program
Formative evaluation is basically concerned with processes. It takes place at the beginning and during the implementation phase of an initiative. The purpose of such evaluation is to identify needs, clarify rationales and improve implementation. In this sense, it is prospective (or forward-looking). It is concerned with continuous assess- ment and ongoing feedback as a means to guide further development of the initiative. If evaluation is continuous, then certain expectations, and assurances, have to be negotiated from the onset of the evaluation project, so that everyone – the funding bodies, the evaluators, the project or programme being evaluated, the client group, the local community – is clear about the role and contribution of evaluation as dis-
tinct from the project or intervention itself.
For continuous evaluation, it is essential that time be spent right at the very begin- ning of the project to have extensive discussions with stakeholders about the evaluation. These preliminary discussions need to be directed at clarifying roles, perspectives and key players, and the specific tasks of the evaluators relative to the overall programme. For instance, who is to collect what data and for what purposes has to be negotiated, and data-sharing procedures have to be put in place. A case study of continuous evalu- ation is provided in the next example, which is introduced in Box 22. 4.
From the point of view of evaluation methodology, it is important to monitor those factors which impact on the evaluation process and which may either enhance or debilitate the evaluation. Building trust relationships is essential, and can be achieved to some extent by ensuring constant and transparent communication among key stakeholders. On the other hand, there is always the danger in continuous evaluation (which requires assessment of progress and regular feedback to projects/ programmes on how best to improve performance or achieve mission objectives) that the evaluators end up becoming de facto project managers. Again, clarity of roles and
 |
TaBle 22. 1 Matters arising and PROP responses
Matters arising Responses by PROP team
Defusing suspicion and resistance to collaboration
Educating all referral agencies about what the PROP project is (and is not), to address underlying fears about duplication of roles and services
Some defensiveness still exists surrounding access to PROP clients and their families for interview. This needs to be surfaced and discussed between PROP case workers and the evaluation team and a sensitive approach taken to working with this vulnerable client group.
PROP has successfully delivered a number of information sessions to Community Corrections staff. The sessions familiarized staff with the PROP progject, its objectives, limitations and referral processes, as well as providing good opportunities for networking and consolidating existing relationships.
|
|
|
TaBle 22. 1 (Continued)
 |
Matters arising Responses by PROP team
Ongoing education and dissemination of information about PROP to referral and support agencies
Technical training for PROP case managers in SMART software
Further streamlining of PROP referral processes with primary and secondary referral sources
Defining key terms such as recidivism for the purposes of this evaluation
For a detailed listing of networking activities, please refer to the PROP Forums, Conferences and Meetings document attached.
The SMART software has been abandoned in favour of a more user-friendly client database in Microsoft Access. Specific training is recommended for the project manager and a back-up person in use of the query/ reporting function of the software to generate monthly reports for the evaluator. The Tasmania Prison Service will provide a resource to assist PROP to achieve this outcome.
A collaborative approach between PROP and TPS is required to achieve a common understanding of what constitutes an ‘appropriate’ referral, together with critique, analysis and documenting of the referral process in both narrative and flow-chart form.
The evaluation has adopted the following definition of recidivism, provided by the policy officer, Community Corrections, at the request of the core working group: ‘return to corrective services with a new correctional sanction within 2 years of release, following a term of sentenced imprisonment, or final discharge from community corrections supervision following completion of an order(s) or supervision requirements’.
This definition excludes ex-offenders returning to corrective services with remand, bail supervision, fine default or fine option matters only (or any combination thereof). Using the Report on Government Services (ROGS) definition will allow the project to make comparisons to published data for all Australian jurisdictions. It should also be noted that there are very clear rules about exactly how this data should be counted and calculated.
Tasmania Together defines recidivism as the proportion of adult offenders convicted again within two years. However, performance data is yet to be published and it won’t allow one to benchmark as widely as ROGS.
Matters arising Responses by PROP team
Clearly defining and documenting the eligibility and selection criteria
Developing a flow chart clearly illustrating how clients will move through the PROP progject step by step
Since inception, the project eligibility and selection criteria appear to have shifted from high risk to medium and high risk. As mentioned above, there is a need for PROP and TPS to collaborate more effectively in defining and refining this criteria, so that there is a common understanding of who is eligible and who is not.
The PROP manager has done some work in this regard and will continue to develop this flow chart over the next six months.
purposes is crucial to distinguishing between project management and evaluation assessment, even though inevitably evaluation feedback will shape the evolution of projects over time.
|
|
|
It also needs to be emphasized that evaluators engaged in continuous assessment need to be independent of the project or programme, and to ‘play with a straight bat’ when it comes to critical observations of process, detailing problems in project man- agement, identifying problems in data collection and more generally acknowledging both the strengths and limitations of project development. At a personal level, the relationship between evaluators and evaluatees can sometimes be close and involve individuals who know each other quite well. Negotiating personal relationships is part of the challenge of providing evaluation that is fair, objective and constructive. Living with the consequences of what one ‘sees’ and describes is one outcome of such processes. These consequences can be very personal and difficult. For instance,
as part of the feedback the evaluation team made the observations in Table 22. 2.
 |
TaBle 22. 2 PROP team observations
|
|
|
