POP Center Tools Assessing Responses to Problem, 2nd Ed Page 2

The Role of Evaluation in Problem Solving

Assessment is the final stage in the SARA problem-solving process.2 It is a systematic attempt to determine whether a problem declined after an effort was made to reduce it. Though assessment is the final stage of the problem-solving process, you will be making critical decisions about evaluation throughout the SARA process. The left side of Figure 2 shows the standard SARA process and some of the most basic questions asked at each stage. It also draws attention to the fact that the assessment stage may produce information requiring the problem solver to go back to earlier stages to make modifications. This is particularly the case if the response was not as successful as anticipated.

On the right side of Figure 2 are critical questions that need to be addressed in order to carry out an evaluation. During the scanning stage, you must define the problem with sufficient precision that it can be measured. Here is where you determine “what success looks like”. What is the minimum necessary problem reduction that is acceptable? At this stage, you will collect baseline data on the nature and scope of the problem. At the analysis stage, you will collect data describing the details of the problem: who is impacted, when, where, and by how much? Virtually every important question to be addressed during analysis will be important in the assessment stage. This is because during assessment you want to know whether the problem has changed: information uncovered during the analysis stage becomes vital baseline information (or “pre-response” measures) for the assessment stage.

During the response stage, while developing a strategy to reduce the problem, you should also develop an accountability mechanism to be sure that various participants in the response do what they should be doing. As we will see later, one type of evaluation—process evaluation—is closely allied to accountability. Also, the type of response used will have a major influence on how the other type of evaluation—impact evaluation—will be designed.

All of these earlier decisions are brought together during the assessment stage to answer questions: Was the response implemented as planned? Did the problem change (decline)? Are there good reasons to believe that the response is the most important explanation for the changes in the problem?

In summary, you begin planning for an evaluation when you take on a problem; the evaluation builds throughout the SARA process, culminates during the assessment stage, and provides findings that help determine whether you should go back and revisit earlier stages in order to improve the response. Appendix C contains a checklist of questions that can be used as a general guide to evaluation throughout the SARA process.