• Center for Problem oriented policing

POP Center Tools Assessing Responses to Problem, 2nd Ed Appendix D

previous page next page

Appendix D: Summary of Evaluation Designs' Strengths and Weaknesses

DESIGN

STRENGTHS

WEAKNESSES

STATIC COMPARISON

  • Better than nothing
  • Can be useful for preliminary examination
  • Pre-existing differences between groups are likely to be the cause of the different levels of the problem
  • Cannot determine if the response came before or after the pre-existing differences
  • Cannot eliminate the possibility that pre-existing trends created the results

PRE-POST

  • Simple and quick to implement
  • Can easily be used with surveys
  • Can provide a reasonable estimate of the change in a problem following a response
  • Can only show short-term changes in problems
  • Cannot account for pre-existing trends
  • Very weak at eliminating alternative explanations for the change in the problem
  • Cannot account for the possibility that some other factor occurred at the same time as the response, and caused the problem to change

INTERRUPTED TIME SERIES

  • Very easy to use with data routinely collected over many time periods
  • Can eliminate pre-existing trends and many other alternative explanations
  • Very hard to use if special data-collection efforts, such as surveys, are used to measure the problem
  • Cannot account for the possibility that some other factor occurred at the same time as the response, and caused the problem to change
  • Results take a long time to be established
  • Difficult to interpret when there are few problem events per time period before the response

PRE-POST WITH CONTROL

  • Can easily be used with surveys
  • Can account for the possibility that some other factor occurred at the same time as the response, and caused the problem to change
  • Can only show short-term changes in problems
  • Requires a problem-troubled control group that will not get the response and is similar to the response group

MULTIPLE TIME SERIES

  • Easy to use with data routinely collected over many time periods
  • Can eliminate pre-existing trends and many other alternative explanations
  • Can account for the possibility that some other factor occurred at the same time as the response, and caused the problem to change
  • Very hard to use if special data-collection efforts, such as surveys, are used to measure the problem
  • Requires a problem-troubled control group that will not get the response and is similar to the response group
  • Results take a long time to be established
  • Difficult to interpret when there are few problem events per time period before the response

previous page next page