What is Evaluation?

 

Evaluation is the organized and systematic process of collecting information to answer a question about a community or program. The question asked may be about how best to identify a need and set goals to address the need; what is the best program intervention that addresses that need; how should it look in a particular community; and what are the markers to let us know the program is accountable and successful. Some actual questions include:

 

·         What is the need in this community?

·         What type of program would best meet this need?

·         How should this program be implemented?

·         What should be included in the program?

·         Is the program doing what we thought it would? If not, why not?

·         Is the community better informed, more aware, healthier, safer, etc., because of this program? How do we know?

·         Have there been any problems or unexpected difficulties or barriers?

·         Has the program been implemented as it was designed?

·         Is it effective in addressing a community need?

·         How effective has it been?

·         What signs or measures have been observed (or what information or data has been collected) to let us know the program is working?

 

These are just a few examples of different types of evaluation questions. Evaluations themselves are an ongoing process that provides a stream of information and feedback regarding a project to continually improve the program?s performance and contribution to the community over time.

 

Benefit and Value of Evaluations

There are many reasons to evaluate a program, the main one being program improvement. They allow a review of prior activities and offer direction for improving future efforts. They involve community members in their own health and healing processes and create program ownership. They increase pride in the community by demonstrating in tangible ways the efforts and successes of the programs. They not only demonstrate success to the community but also to program grant makers and lay the foundation for continued or expanded funding opportunities.

 

Evaluations must be approached from the perspective of assisting a program, rather than punishing a program. Information gathered in an evaluation serves to guide the next phase of the project ? the information isn?t ?good? or ?bad? it is just information. If a program does not achieve the original goals or objectives, the evaluation can document the efforts and explain the results. It can identify unanticipated barriers or delays and help to formulate a plan to address them. Too frequently, program staff are resistant to the evaluation process for fear of punishment or blame if there are problems. Once staff members realize their work can be improved and their efforts can make a difference, they become advocates for the evaluation activities.

 

What are Goals and Objectives

Program evaluation cannot occur without clearly defined goals and objectives. The objectives, in particular, provide the criteria for evaluation. Before a project intervention is designed and implemented, the program goals and objectives must be clearly identified, measurable and stated in writing. This is important because these goals and objectives serve as a road map for work activities to be done during a project period and as a standard for use in measuring success. Outcome and process objectives provide a quantitative measurement of change that the program can and should accomplish by some future date.

 

Goals

A goal is a general statement of what your project hopes to accomplish, or to contribute toward accomplishing, in the long term. It seldom has a deadline attached, and may not be achievable through the efforts of a single project, although several projects or efforts together should contribute towards the goal. Often progress towards a goal can be determined only after some years of effort. A goal should be challenging but attainable.

 

Objectives

Objectives are the short-term or intermediate accomplishments that contribute to the attaining the goal. They can be measured during the project period. Objectives are:

 

·         Specific: Identify whom, what and where.

·         Measurable: Identify when and how many.

·         Reasonable: Can be achieved within a specific time period using available technology and resources.

 

Types of Objectives

 

1.       Outcome Objective

An outcome objective states the desired results of a program or intervention in measurable terms. It must state a desired result in measurable terms with a time deadline; it should be specific, realistic, and achievable during the project period.

 

 

 

 

Outcome objectives are usually measured by:

·         Levels of mortality, morbidity and/or disability

·         Levels of health conditions (HIV)

·         Behavioral measures (IV drug use)

 

2. Process Objective

A process objective describes key program activities or tasks essential to achieving an outcome; taken as a group, process objectives are steps along the way towards the desired results. A process objective must be specific, realistic, measurable and time-phased.

 

Interventions can include:

·         Health services

·         Health education

·         Counseling

·         Regulatory actions

·         Legislative and policy changes

 

 

Process objectives are:

• Short-term (usually one year)
• Realistic

• Measurable

 

There must be a logical, practical relationship between the outcome and process objectives. This relationship is based on educated projections of how much of what types of interventions will results in the expected or desired change in the identified health problem. For any one outcome objective, there may be several process objectives. That is, there may be several different interventions that all lead to the same desired change in health status.

 

 

 

Well-written Outcome and Process Objectives

All outcome and process objectives must include certain components to make them useful tools for measuring changes in health problems and interventions such as?.

 

Component	Outcome Objective	Process Objective
When	The time (month, fiscal year, calendar year) by or during which the change in health status would be achieved.	The time (month, fiscal year, calendar year) by which the intervention should be accomplished.
What	The targeted health problem or health behavior to be decreased, increased or maintained.	The targeted intervention (health service, health education, counseling, regulatory action, or legislative/policy change) to be accomplished.
Whom	The target population who will benefit from the change in health status.	The target population that will benefit from the accomplishment of the intervention.
Where	The area in which the target population is located (city, county, clinic).	The area in which the target population is located (city, county, clinic). Or where the intervention will take place.
Who	The staff or agency responsible for correcting the health problem.	The staff or agency responsible for carrying out the proposed intervention.
How Much	The quantity of change in a health problem.	The amount of the intervention to be utilized, performed, or accomplished.

 

When is expressed as a fiscal year (local, state, or federal) or calendar year. Whether fiscal year or calendar is used often depends on how the data is collected ? vital statistics are collected on a calendar year basis, whereas clinic visit data may be collected by calendar or fiscal year. Units of time smaller than one year are used only in activities.

 

Who is not always spelled out in objectives, particularly in outcome objectives, if the responsible staff or agency is self-evident or is later defined in process objectives or activities. The other five elements are usually included.

 

Preferably, how much is written as a target rate, a percentage or a specific number, with the current rate, percentage, or specific number included for reference purposes.

 

With the increasing emphasis on accountability and performance measurement, objectives must include a measurable component and the data to measure the objective must be available. In reviewing objectives you have written, you may find that you do not have the data sets or data collection systems to measure some of the objectives. You may have to rewrite some of the objectives to reflect what you can measure. Afterward, you can work on developing the data collection systems pertinent to what you need to be able to measure in the future.