POP Center Tools Analyzing and Responding to Repeat Offending Page 3

Assessing the Effectiveness of Strategies to Reduce Repeat Offending

It is important to assess the impact of measures you implement to reduce repeat offending to determine if your responses have been effective or need improvement. You should consider assessing your responses from the outset, not just after you’ve initiated them. Measurements should be made before you put responses in place, to determine the extent and seriousness of the repeat offending problem, and after you implement them, to measure their effectiveness. To do this, you need to focus both on process measures and outcome measures.† Putting in place an effective strategy to reduce repeat offending is tricky, and it is important to track your activity throughout. Particular matters to focus on are determined, of course, by the specific problem, your understanding of it, and the measures you have taken to reduce it. In general, however, you need to consider the following questions in the process element of your assessment:

1. How were the repeat offenders identified, and how were any replacements identified?
2. How was it decided that a repeat offender no longer warranted continued attention?
3. What was implemented, and where and when?
4. What partner agencies were involved in the implementation? Did they deliver the agreed-to measures as planned?
5. What adjustments were needed to make the initiative run smoothly?
6. What other changes took place that might have affected the particular problem (for example, unplanned publicity)?

† See Problem-Solving Tools Guide No. 1, Assessing Responses to Problems: An Introductory Guide for Police Problem-Solvers for further information. Also see Tilley (2009) and Knutsson and Tilley (2009). 

When measuring outcomes, you must be wary of the various ways in which you might be tempted to draw misleading conclusions. These include the following:

1. Regression to the mean: focusing on offenders when they are committing crime at their highest rate. Crime is likely to fall back to a more normal rate without intervention, but you may mistakenly conclude that it follows from your efforts.

2. External factors: Causes that are unrelated to your initiative may affect the rate of offending more generally; for example, a change in the law or the introduction of new services independently of the specific problem-solving efforts or the introduction of crackdowns following political pressure.

3. General trends: There may be underlying trends in repeat offending patterns that mean any observed changes would have happened regardless of any problem-solving intervention.

4. Selection effects: Those subject to treatment may be especially open to change and may not be representative of all repeat offenders. Indeed, they may even change without any particular measures being applied to them.

5. Methods of recording data: These may alter in ways that create the appearance of change where there is none. Methods of counting crimes change. Policies on arrest may change. Data recording practices may be altered. Any of these may affect counts of offenses and offenders that could influence outcome assessment findings independently of any change in underlying crime patterns.

6. Methods of identifying offending: Discretion may be used in processing offenders in ways that affect counts; for example, treatment is being given which may mean more reoffending comes to light or there is a temptation to overlook offenders’ lapses in the interests of maintaining client participation.

To assess outcomes, you need to work out the expected pattern of effects and check whether they are occurring. For example, if your project is attempting to incapacitate known or suspected repeat offenders, you need to track who is and who is not at liberty in the community to offend each day, to determine whether rates of the reported crimes being targeted are affected by the availability of the suspected repeat offenders. On the other hand, if you are trying to bring into treatment those deemed liable to be prolific offenders, you should look at before and after rates of reported relevant crime, controlling for the supply of those being targeted. If your project focuses on a rash of offenses attributed to one offender, then a dramatic fall in the crime rate would be expected once that offender
is taken into custody or effectively deterred. If the project targets those leaving prison, then before and after rates of re-arrest for the population of those released may provide a suitable measure of impact.

When conducting your evaluation, look for positive and negative side effects as well as whether the intended outcomes occurred. It is difficult to anticipate unintended consequences, but they can be important. Displacement (the substitution of some alternative offending for that which is prevented) and diffusion of benefits (the extension of crime prevention effects beyond the direct application of measures) are frequently considered side effects of problem-solving measures. You should consider where these might be most expected and implement efforts to capture them.† For example, if you incarcerate prolific drug dealers in an effort to incapacitate them, it is important to determine
whether they are replaced by other drug dealers. Likewise, if members of high-crime families are targeted, consider an assessment focusing on whether the criminal activity of other non-targeted members is also reduced.

† See Problem-Solving Tools Guide No. 10, Analyzing Crime Displacement and Diffusion, for further information.