Highlighted Program Evaluation Resources

Developing a Logic Model

  1. Learning From Logic Models in Out-of-School Time. Harvard Family Research Project 11 page s . 2002. English . http://www.hfrp.org/publications-resources/browse-our-publications/learning-from-logic-models-in-out-of-school-time

    Presents the uses, benefits, and practical applications for a logic model as a method for presenting the design of Out-Of-School Time (OST) programs in order to assess progress. A logic model uses one sheet of paper to summarize a program's key elements, approach rationale, desired outcomes and measurements, and cause-and-effects relationships between a program and its intended outcomes. The left side of the model describes the 4 program elements that describe the OST program: (1) desired results, (2) motivating conditions and causes, (3) strategies, and (4) activities. The right side of the model lists the OST program's outcomes, performance measures, and indicators. An additional element, evaluation and learning, can be added to assess progress and plan improvements. A guide to terms and definitions, a glossary, an example, and a model worksheet are provided.

  2. Learning from Logic Models: An Example of a Family/School Partnership Program. Coffman, Julia page s . 1999. English . http://www.hfrp.org/publications-resources/browse-our-publications/learning-from-logic-models-an-example-of-a-family-school-partnership-program

    This brief offers a step-by-step approach for developing and using a logic model as a framework for a program or organization's evaluation. Its purpose is to provide a tool to guide evaluation processes and to facilitate practitioner and evaluator partnerships. The brief is written primarily for program practitioners, but is also relevant and easily applied for evaluators.

  3. Logic Model Builder. FRIENDS National Resource Center for Community-Based Child Abuse Prevention (CBCAP),National Clearinghouse on Child Abuse and Neglect Information (NCCANI) . English . http://toolkit.childwelfare.gov/toolkit/

    A logic model is a "map," a simple illustration of what you do, why you do it, what you hope to achieve, and how you will measure achievement. It will include the outcomes of your program, indicators of those outcomes, and measurement tools to evaluate the outcomes.  This Logic Model Builder will take you step-by-step through the process of developing a logic model so you can plan program evaluation activities for child abuse and neglect prevention, family support, and parenting programs. 

  4. Logic Model Resources. University of Wisconsin-Extension page s . var.. English . http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html

    This Web site provides information on logic models, such templates for creating one, examples, a self-study module, training guide, and more. 

  5. Logic Models in Out of School Time Programs: What are They and Why are They Important?. Hamilton, Jenny , Bronte-Tinkew, Jacinta 6 page s . January 2007. English . http://www.childtrends.org/?publications=logic-models-in-out-of-school-time-programs

    Logic models are powerful tools for designing, planning, implementing, and evaluating out-of-school time programs. This brief describes the key components of a logic model, identifies why logic models are useful, discusses different types of logic models, the formats that they take, and the resources available to programs for creating them. Logic models also are called conceptual models and theory-of-change models. - Publisher's description

  6. Measuring Program Outcomes Training Kit. United Way of America 72 page s . 1996. English This resource may be free from your local library or purchased from the publisher.

    Provides practical guidance for trainers who are engaged in helping to prepare health, human service, or youth- and family-serving organizations to identify and measure their program outcomes. The training kit contains modules that address the many steps involved in planning and implementing an outcome measurement system. The modules cover: (1) basic concepts in outcome measurement, including the logic model shuffle, as well as the limitations of outcome measurement; (2) first steps in assembling an outcome measurement workgroup, including deciding on a target program and developing a timeline; (3) constructing a logic model for the program outcomes that will be measured; (4) identifying outcome indicators; (5) identifying data sources for the selected indicators, designing data collection methods, and pre-testing data collection instruments and procedures; (6) developing a trial strategy, tracking and collecting outcome data, and monitoring the outcome measurement process; (7) entering and tabulating data and analyzing and reporting findings; (8) improving the outcome measurement system; and (9) understanding how data can be used to improve programs. The kit also includes: tips on how to wrap up each training session, camera-ready originals of worksheets and handouts to use during training, and a short video that defines basic concepts in program outcome measurement.

  7. The Logic Model for Program Planning and Evaluation. McCawley, Paul F. 5 page s . n.d.. English . https://www.cals.uidaho.edu/edcomm/pdf/CIS/CIS1097.pdf

    This article provides an overview of logic models, including information on why they are used and how to create one.  It provides a diagram of the elements of a logic model, including the differences between short-, medium-, and long-term outcomes.

Developing Outcomes and Indicators and Choosing Measures

  1. Afterschool Evaluation 101: How to Evaluate an Expanded Learning Program. Harris, Erin 44 page s . 2011. English . http://www.hfrp.org/out-of-school-time/publications-resources/afterschool-evaluation-101-how-to-evaluate-an-expanded-learning-program

    Afterschool Evaluation 101 is a how-to guide for conducting an evaluation. It is designed to help out-of-school time (OST) program directors who have little or no evaluation experience develop an evaluation strategy. The guide will walk you through the early planning stages, help you select the evaluation design and data collection methods that are best suited to your program, and help you analyze the data and present the results.  (Description from source)

  2. Evaluation Toolkit. FRIENDS National Resource Center for Community-Based Child Abuse Prevention (CBCAP) page s . n.d.. English . http://friendsnrc.org/evaluation-toolkit

    Supports social services program managers and administrators with plans to develop evaluation tools in order to assess program effectiveness and efficiency. The Federal Government is demanding strict outcome accountability criteria for social services programs, especially for child abuse prevention. This Four Step Toolkit can assist in the development of a cyclical and evolving evaluation system. Step one provides Internet links to numerous websites in order to develop a strong foundational understanding of the outcome accountability process. Step two discusses the Logic Model Builder, developed in collaboration with the National Clearinghouse on Child Abuse and Neglect Information, which allows users to input outcomes and indicators to a template that can be used for their own program. Step three, Outcomes and Indicators, provides short-term and intermediate/long-term outcomes and indicators on the program goals' five domains: child and family health, parenting skills, child development, family relationships, and formal and informal sources of support. Each of these outcomes and indicators within the domain may be edited to reflect the intent and scope of the local program. Step four, Annotated Measurement Tools, provides a compendium of popular tools used to measure outcomes in 38 prevention programs. A link leads to additional discussion on the construction of evaluation tools. CONTENTS 1. Building Your Evaluation Plan 2. Logic Model Builder 3. Outcomes and Indicators 4. Annotated Measurement Tools

  3. Measurement Tools for Evaluating Out-of-School Time Programs: An Evaluation Resource. Brosi, Evelyn page s . 2011. English . http://www.hfrp.org/out-of-school-time/publications-resources/measurement-tools-for-evaluating-out-of-school-time-programs-an-evaluation-resource2

    This guide describes a select set of instruments and tools that can be obtained and used for on-the-ground program evaluation. Whether you are conducting first-time internal evaluations or large-scale national studies, these evaluation instruments can be used to assess the characteristics and outcomes of your programs, staff, and participants, and to collect other key information. (Description from source)

  4. Performance Measures in Out-of-School Time Evaluation. Little, Priscilla M.D. , Harris, Erin , Bouffard, Suzanne 7 page s . March 2004. English . http://www.hfrp.org/publications-resources/publications-series/out-of-school-time-evaluation-snapshots/performance-measures-in-out-of-school-time-evaluation

    This report is a compilation of resources from the OST Program Evaluation Database and provides the academic, youth development, and prevention performance measures currently being used by OST programs to assess their progress.

  5. PerformWell (Web site). Urban Institute, Child Trends, & Social Solutions page s . 2012. English . http://performwell.org/

    This Web site provides information on performance management strategies and can be used by social service organizations to deliver more effective social programs.  It allows programs to identify their outcomes and then choose indicators and tools, such as surveys or assessments, to determine when their outcomes have been reached.

  6. What's Working? Tools for Evaluating Your Mentoring Program. Saito, Rebecca N. 136 page s . 2001. English . https://www.searchinstitutestore.org/product_p/e407-w.htm

    Describes evaluation methods that give mentoring program providers a reliable way to assess the outcomes and benefits of participation in their programs. The evaluation kit contains focus group and interview questions, surveys, and related tools that address the effect of mentoring relationships on youth, the extent to which these relationships meet critical quality benchmarks, and how programs can be improved. Mentoring program providers get step-by-step guidance on: (1) planning their evaluation and choosing the appropriate tools, including the pros and cons of open- and closed-end question structure and mail versus in-person survey administration; (2) conducting the evaluation by means of the selected tools, including different approaches for mentees and mentors, tips for facilitators and note takers, and tallying techniques; (3) understanding and communicating findings, including key components of the evaluation report and extrapolating lessons from key results; and (4) criteria for selecting outside evaluators. With evidence of the benefits and outcomes of mentoring programs, policymakers will have the information necessary to ensure adequate support for this promising strategy for positive youth development and community change. Also contains a list of youth developmental assets as well as numerous reproducible forms, tally sheets, summarized theme analyses, and overall program rating sheets.

Choosing Interventions

  1. Culturally Appropriate Programming: What Do We Know About Evidence Based Programs for Culturally and Ethnically Diverse Youth and their Families?. O'Connor, Cailin , Small, Stephen A. , Cooney, Siobhan M. 5 page s . January 2007. English . http://fyi.uwex.edu/whatworkswisconsin/files/2014/04/whatworks_01.pdf

    This publication summarizes the growing body of research that provides insights into the cultural appropriateness of evidence-based programs for the prevention of juvenile delinquency and other youth problems.

  2. Evidence-Based Practices for Children Exposed to Violence: A Selection from Federal Databases. U.S. Department of Justice 27 page s . 2011. English . https://www.futureswithoutviolence.org/userfiles/file/Children_and_Families/Complete%20Matrix%20Booklet%2011FEB02.pdf

    This resource summarizes findings and evidence from federal reviews of research studies and program evaluations to help localities address childhood exposure to violence and improve outcomes for children, families, and communities. (Description from source)

  3. Model Programs Guide (MPG). Office of Juvenile Justice and Delinquency Prevention,Development Services Group page s . English . http://www.ojjdp.gov/mpg/

    This Web site provides a searchable database of evidence-based programs across the spectrum of youth services, including family therapy, leadership and youth development, mentoring, parent training, residential treatment centers, and wraparound services. The MPG was designed to assist practitioners and communities in implementing evidence-based prevention and intervention programs that can make a difference in the lives of children and communities.

  4. National Registry of Evidence-Based Programs and Practices (NREPP). Substance Abuse and Mental Health Services Administration (SAMHSA) page s . English . http://www.nrepp.samhsa.gov/

    This resource is a searchable database of evidence-based practices in prevention and treatment of mental health and substance use disorders. The NREPP allows users to narrow their search based on target populations, service settings, and desired outcomes.

  5. Promising Practices Network (Web site). Rand Corporation page s . n.d.. English . http://www.promisingpractices.net/

    The Promising Practices Network (PPN) is a group of individuals and organizations who are dedicated to providing quality evidence-based information about what works to improve the lives of children, families, and communities.  The site helps decisionmakers understand what approaches and programs have been shown in the scientific literature to improve outcomes in various areas such as child health and education.

  6. The Program Directory. Helping America's Youth page s . n.d.. English . http://youth.gov/evidence-innovation/program-directory

    This resource provides information about program designs that successfully deal with risky behaviors, which can then be replicated to meet local needs. It contains risk factors, protective factors, and programs that have been evaluated and found to work.