Types of Data

 

There are different types and forms of data. Quantitative data can be counted - they are numbers such as epidemiological statistics, numbers of people in a community, percentages within different age groups or household income categories. They are the ?bones? of evaluation and build the framework of the evaluation plan and report. Qualitative data is more subjective and ?fleshy.? They put a face and form on the quantitative skeleton.

 

Qualitative data are collected through open-ended questions, written or oral, and through interviews and active listening in focus group situations. It is particularly important when working in a Native community to listen to the responses and gently probe for underlying factors. When evaluating responses, it is also important to differentiate whether you are dealing with primary or secondary data.

 

Primary data are information that is ?first-hand.? It has usually been collected by the program team or the evaluator and may be specific to the topic at hand.

 

Secondary data are ?second-hand? information that is relevant to the project but may have a broader scope. For example, epidemiological data collected by the local or state health department can provide information about sexually transmitted diseases, ages of women at childbirth, numbers of individuals with HIV or AIDS. Information from the Department of Justice can provide information on the demographics of incarcerated individuals. Statistics from the local Council on Alcohol and Drug Abuse can inform programs about the demographics and types of services most utilized. Other sources of secondary information are census records, cancer and tumor registries, mortality and morbidity reports and studies tangential to the project that may have been conducted through local universities. Being mindful of all the different aspects of a project and how it is touched and touches the community may suggest additional resources for secondary data. This type of information is especially important when conceptualizing and developing a program plan. It forms the foundation for the formative evaluation.

 

Work Plan

A work plan is a listing of what tasks need to be accomplished, by whom, by when, and how documented. This can look similar to the logic model with different categories. The work plan for an HIV education program in schools would look something like this.

 

Activities	Action Steps	Responsibility	Documentation	Timeframe
Teachers are trained to deliver the HIV education classes	Training materials are ordered.	Health educator.	Number of invoices for materials.	2 weeks
	Teachers are scheduled for one day of training	Health educator and school nurses.	Number of classes scheduled.	1 month
	Training is conducted.	Health educator and school nurses.	Sign-in logs will reflect the number of classes held and number of participants.	1 month
	Classroom materials are provided to teachers.	Health educator	Records will reflect the number of materials distributed to each school.	Ongoing

 

 

Types of Evaluations

There are different types of evaluations to be used in a variety of settings. Each can be more or less comprehensive, based upon the needs of the program and skills of the program staff.

 

Needs Assessments ? A needs assessment is typically the initial process of determining weaknesses, problems, or gaps in services. While needs assessments are not evaluations, they use the same methodologies and processes for learning about the community. They usually differ in size and scope and lead to subsequent decisions about funding allocations toward areas of greatest need. Much of the information learned in the needs assessment is used in the formative evaluation process.

 

Formative - A formative evaluation is best done before a program design has been finalized and is used to help develop detailed, specific activities to improve a program. It includes collecting all available and existing information about the community or problem to help determine what is already known and helps to identify existing resources. For existing programs, it requires becoming familiar with all aspects of the program; it?s operation, and staff responsibilities.

 

This process of finding out important information from already existing sources is called collecting secondary data. For example, if a Native urban clinic has noticed an increase in teens with a sexually transmitted disease, and knowing that STDs are co-factors for HIV disease, the clinic might want to start an HIV awareness program targeting teens.

 

In conducting a formative evaluation prior to implementing the program, the clinic staff could collect data from their own records to measure the change in treatment seeking, talk with the local health department to find out current epidemiological statistics on STDs in teens, talk with other clinics and providers to see if this was a common phenomenon, seek out information from the local community planning group?s HIV prevention plan, talk with high school nurses, and others working with teens to gain additional perspective on what could be contributing to the situation . This formative evaluation process will then give the program planners either new information, or facts and figures to support their basic assumptions.

 

Process - A process evaluation monitors how a program evolves and moves forward - the process of the program. It identifies what is working and not working, how a program is being accepted or perceived by the community, and provides warning signs when a program may be going off track. A process evaluation can include collecting both quantitative data and qualitative data. For example, to evaluate a training program, the survey might collect quantitative information such as the demographics about participants and pre-post test knowledge. Qualitative questions might ask about their perceptions of the training, remaining questions, and level of satisfaction.

 

Summative ? An summative evaluation recounts and presents information needed for summary statements about the value of the program and it is used to decide if a program should be continued or dropped. It reports on the general activities of the program, describing what services were offered, to how many people. A summative evaluation compares and contrasts the results of a program. This may be a comparison against the results of former years, a program with the same goal, or results from standardized tests. For example, an outcome of an HIV health education program may be that 25 Native women attended a 3-part training program as compared with 15 participating in the prior year; or that 50 people participated in a condom demonstration, down from 150 in the prior year; or that scores on a standardized pre-post test on HIV education are higher than the group norm. In each of these examples, it is easy to report on the activities, to count the encounters. Demonstrating that these efforts changed a behavior falls under the term ?impact? evaluation. Often these types of evaluations are organized and researched at the conclusion of a program. While it is important to understand the activities and history of a project, it is much more beneficial to the program staff, and ultimately the community, to conduct ongoing evaluation activities that provide feedback and offer insightful direction.

 

Outcome - An outcome study shows how well the goals and objectives of a program were achieved, usually on a short-term basis. For example, a quantitative outcome study using a pre-post test survey would be implemented after a HIV education program to see how much knowledge had changed. Another outcome measure might demonstrate how many people participated in an HIV prevention training. A qualitative outcome study would be able to discuss how perceptions of a program had changed.

 

Impact ? An impact evaluation extends the outcome reporting efforts beyond outcomes by reporting on the effect the program had on the population and discussing the results of the program - how we know we made a difference. Impact evaluations are usually long-term (6 to 12 months) and seek to answer questions about the long-term impact of the program. Some study questions for an impact evaluation of HIV prevention programs might be:

 

• How do we know people are using condoms more? How can this change be documented over time?
• How do we know people are comfortable in discussing sex with their potential or existing partners? How has our HIV education program helped them gain these skills? How can this be documented?
• Are adolescents delaying sex? How do we know?
• Are intravenous drug users using bleach kits and using clean needles? Ultimately, is the infection rate declining?