What we measure matters. But how we measure programs and policies is critical, too. People are at the core of any data point—mothers, fathers, children, our friends, and neighborhoods.  Abt Global is committed to ensuring that equity-related principles and practices are meaningfully integrated across all phases of our evaluation work. If there were a single principle to embrace, it would be nihil de nobis, sine nobis. Translated as “nothing about us without us,” this sentiment conveys that no policy should advance without the participation of those who will be affected by it.

This absolutely holds true for program evaluation: active community participation must be central to and integrated throughout the evaluation life cycle, because equity is about shifting the locus of power and focus, which means sharing that “power” by including in the process those who will be impacted. With that in mind, here are some thoughts about seven aspects of evaluation, from teaming-up and evaluation design through final reporting.

 1. Equity-Focused Evaluation Starts with Inclusivity.

• The evaluation team should include people with diverse perspectives and experiences to ensure the authentic integration of those perspectives into practice.
• Even beyond direct team membership, the evaluation design, research questions, logic models, outcome metrics, and analysis and reporting plans should actively integrate the voices and lived experiences of systematically marginalized persons and communities that are impacted by the work.

2. Formulate Equity-Focused Research Questions.

In addition to addressing the overall program impact, evaluation questions should ask:

• Who do programs benefit the most and least?
• How can we best generate evidence to gain insight on these disparities?
• How should we measure unintended consequences that could exacerbate inequities?
• What policies, systems, and environments influence program impacts and how?

3. Design the Evaluation with an Equity Focus.

• Design control or comparison groups to target key characteristics of systematically marginalized persons and communities, both in general and to permit subgroup analyses. Consider using tailored, possibly separate, comparison groups.
• Use measures that explicitly capture both equity outcomes and the drivers of them. In a given context, consider: what does improving equity—and the drivers of it—look like?

4. Collect Data with an Equity Focus—on Both Process and Measures.

• Incorporate the voices of systematically marginalized persons and communities in data collection activities.
• Collect primary baseline data from key populations and subgroups against which to measure change. In the process of collecting data, consider:
  • Instruments: Engage diverse populations when developing your instruments, and assess bias in already-validated instruments.
  • Outreach and recruitment: Implement strategies to engage diverse populations and promote high response rates.
  • Burden: Ensure that the process of data collection does not have an inequitable burden across research participants.
  • Ability to detect impacts: Use oversampling to ensure that your sample size will be sufficient to assess various populations of focus.

5. Advance the Necessary Analytic Approaches.

• Have a deliberate and thoughtful approach for handling missing data, including:
  • Understanding whether data are missing because of inequities; and
  • Using nuanced procedures to handle missing data that is otherwise challenging to impute.
• Advance new approaches for addressing disparities between groups where commonly used ones (for example, underpowered subgroup analyses) are insufficient.

6. (Re)define “Success.”

• Go beyond program-wide results and:
  • Highlight changes in outcomes for key subgroups.
  • Measure the characteristics of participants who were successful or unsuccessful in a program, with consideration of the influence of external factors.
  • Assess the characteristics of programs that are particularly successful (or unsuccessful) at serving diverse populations or specific subgroups.
• Consider the concept of “cost-benefit” broadly. Evaluations that are fully equity-attentive may cost more, both financially and in staff time, but they also stand to produce better results, which is a valuable return on the investment.

7. Equity-Focused Evaluation Ends with Inclusivity.

• When interpreting, framing, and messaging results, researchers must be purposeful in incorporating the voices of systemically marginalized persons and communities.
• Share evaluation results with those same people, as well as with other stakeholders.

These seven areas of consideration are necessarily high level. The work required to carry out equity-focused evaluation in practice is detailed and painstaking, but it will be rewarding.

Secondary Data
While beyond the scope of this paper, it’s important to note that action is needed to improve existing data sets, including identifying gaps where data collection and better, nuanced measures are sorely needed. Without good data that includes intersectionality metrics and considers the social determinants of health, the quality of evaluations will be compromised.