Keeping Tabs on Progress

The last step in the problem-solving process is not a single step, but an on-going program of monitoring and evaluation. Evaluation is an on-going activity because change sometimes occurs in small increments so that measurable improvements take a long time to emerge; because immediate change may be an outcome of engagement in the process that disappears over time as interest and attention wane; because the improvement plan is likely to include short-term projects as well as long-term investments, and all of them must be evaluated; and because programs and strategies will need to evolve as environmental conditions change.

The purpose of the evaluation is to decide whether:

• the problem has been eliminated, either temporarily, or permanently
• the problem is occurring less frequently
• the impact of the problem has been reduced (i.e., fewer victims, less violence, smaller losses)
• fewer problems are noted in areas adjacent to the problem location
• the problem has moved to another location
• a new or different problem is emerging
• the problem remains, unchanged.§

§ Goldstein (1990) and Eck and Spelman et al. (1987) include "the problem has been successfully removed from police consideration." Because CPTED engages a variety of organizations and agencies on a problem-solving team—including police officers—the problem may never actually be "removed" from police consideration, even if it becomes the responsibility of another team member.

If the assessment shows the problem has been eliminated or reduced in its frequency or severity, then no additional measures are necessary. The other results, though, suggest it is time for a new problem-solving process linked to new or different outcomes and approaches.

An earlier section of this guide outlined eight categories of data that are used to establish goals and indicators of success linked to those goals. Many police programs rely on indicators such as crime, victimization and fear, or response times or clearance rates. These measures continue to be important, but other indicators may prove equally useful depending on the problem, the setting and the circumstances. A return to the eight categories of data offers some perspective on the options available. Each is discussed in greater detail below, including an estimated time before noticeable and measurable change might be evident.

Crime data. In most cases, reductions in calls for service and reported crime are the goal; however, this is not true in all instances. Some communities may instead be working toward an improved relationship with police, or for greater participation in programs like Neighborhood Watch. Increases in calls for service or reported crime are legitimate outcomes under those circumstances.

It is also possible that the number of incidents will not decrease, but the types of incidents that take place are less violent, involve fewer victims, and result in fewer losses, leading to the perception that conditions have improved.

Alternatively, the evaluation may show that the distribution of incidents has changed either temporally or spatially. These changes may mean the crimes are more easily observed, that the police agency can respond more quickly, or that there are fewer complaints about the problem.

It is also possible that when strategies are successfully implemented at the problem site or location, areas surrounding the site also experience reductions in crime. These types of circumstances suggest the need for broad geographic coverage during both data collection and during the evaluation.

Population characteristics. A neighborhood improvement program may be focused on increasing the diversity of residents with regard to age, gender, race, ethnicity or income; creating a more stable population base indicated by an increasing number of family households; improving resident quality of life by increasing household income; or establishing an enclave for a specific racial or ethnic community; etc. But in some cases the goal may be to support the existing population and see that its characteristics do not change.

Institutional and organizational relationships. Indicators of success in this category might include active community groups with widespread participation; an increase in the number of associations/organizations/institutions working with the community; an increase in property investment; or an increase in support services targeted to residents. Each of the support services may have its own set of indicators—and the organizations involved should participate in, or be linked to, the CPTED evaluation.

Land use and development patterns. Land use and neighborhood stability are very much related. Indicators of stability include:

• constant, or increasing, property values and rental rates
• a higher proportion of owner-occupied property (rather than rental property)
• fewer vacant lots, dwelling units or commercial spaces, and/or increases in construction or rehabilitation activity
• a more compatible mix of uses, or a more diverse mix of uses
• fewer building, fire, health and zoning code violations
• reduced turnover time (time for property to sell or rent)
• increasing contributions of taxes or fees.

Traffic, transportation and transit systems. Speeding and traffic enforcement are common issues in problem neighborhoods. Evidence of increased enforcement, through the number of citations issued, leading eventually to fewer complaints about speeding problems, is one possible indication of improvement.

When a plan includes changes to traffic patterns through street closings or traffic calming measures, indicators of success are required for the target neighborhood, and also for surrounding communities that may be impacted by new travel patterns.§ This can include numbers of complaints or numbers of accidents, changes in traffic volumes or turning movements, etc. Alternatively, the evaluation may consider the number of pedestrians, bicyclists or others using sidewalks, trails and greenways.

§ For further information, see the Problem-Specific Guide on Speeding in Residential Areas and the Response Guide on Closing Streets and Alleys to Reduce Crime. 

Transit ridership is an important aspect of that system's successful operation. Real and perceived safety during travel to and from stops or while waiting for or riding the bus or train can be critical. Increased ridership, a more diverse user population, and ridership that is more distributed geographically may be indicators of a successful campaign to improve transit safety. Alternatively, the goal may simply be to increase ridership and the perception of safety for one transit stop or along one route.

Resident or user surveys, and stakeholder interviews. Reductions in fear and victimization are critical, but are not the only opportunities for improvement in this category. For example, one goal for the program might be a better relationship with police, so that increased reporting of victimization, or greater cooperation during investigations, are the ideal outcomes. Additionally, look for changes in activities and schedules showing that people are less afraid to use various places and spaces, or an improved opinion about the overall quality of life in the community.

On-site behavioral observations. The evaluation should show a reduction in problem behaviors and more widespread activity by a critical mass of "good" users. As with other categories, greater diversity with regard to age, race, income, etc., can be important.

Safety audits and security surveys. Follow-up safety audits and security surveys should reveal that critical recommendations have been implemented. This allows for testing or evaluation of the results of those implementation activities, which might include changes to policies and procedures such as key control; modifications to building layout or site landscaping; additional security measures like locks or CCTV; etc. For example, one indicator of success might be better record keeping, resulting in better information and a quicker and more targeted response to emerging problems.

What should become clear from this summary is that indicators of success absolutely must be tied to program goals, because different goals equate to different results for some measures. What is also clear is the need for quality data collection and analysis during the early phases of problem-solving, so that baseline measures are available and the data afford an opportunity to understand the true impacts of program implementation.

The problem is that evaluation is frequently ignored, overlooked or under-appreciated. Three possible reasons for this are:

1. For many participants, the goal of the problem-solving process is to "do something." Once a program, project or strategy is in place and underway, they are satisfied. They see the process as complete.
2. Evaluation can be time consuming and costly. Other tasks such as those related to implementation, are given higher priority.
3. Problem-solving using crime prevention through environmental design can result in multiple programs or projects. In a setting where many other circumstances and conditions are constantly changing, it is often difficult to determine which changes are the outcomes of specific CPTED initiatives or of CPTED generally, and which changes are produced by other factors in the environment.

Given its role in the problem-solving process, evaluation is an essential and valuable tool for decision making. It affords an opportunity to understand what is working, where it is working and why it is working (or is not working). Evaluation aids in recognizing change, and information from one evaluation can be used as part of the problem-solving process somewhere else. This means that data collection and analysis need adequate time and attention early in the process, and evaluation needs adequate time and attention later on.

Additional information on methods for data collection and evaluation are available at the POP Center website: www.popcenter.org.§

§ For example, see Crime Analysis for Problem Solvers in 60 Small Steps and Assessing Responses to Problems: An Introductory Guide for Police Problem-Solvers.