In Health Systems Strengthening, Evidence Is Elusive. CAMEL Principles Provide New Thinking.
In the classic story of Moby Dick, Captain Ahab obsesses over catching his white whale. It’s all he can think about, his fixation, and eventually, this obsession consumes him, and the whale drags him down into the ocean. While health systems practitioners aren’t going out to sea with men named Ishmael, we chase our own white whale: evidence. Evidence that health systems interventions impact the availability, affordability, and accessibility of healthcare.
Yet, proving the link between an intervention and health outcomes has proven as elusive.
Evidence that a policy, system, or structure works in one setting does not mean it will work in another. Understanding the connections between multiple interventions occurring in different contexts—to attribute outcomes to specific actions or identify causal pathways—is also a chronic challenge for health system evaluators.
Amid these difficulties, awareness of complexity supports understanding the connection between health systems interventions and health outcomes— a key goal of health systems research. I recently moderated a panel at the CORE Group’s Global Health Institute on new ways of framing evaluations through a complexity-aware lens, a necessity not only for finding evidence but also knowing how to use it to improve systems and outcomes
Here are four new insights that could help health systems practitioners and researchers understand how complex health systems work, so that maybe, unlike Ahab, we can get a bit closer to catching our own white whale.
Complexity awareness brings together existing approaches under a new set of principles.
For years, evaluation specialists have used methods like Outcome Harvesting, Ripple Effects Mapping, and Contribution Analysis to understand complexity (see box). While these methods have complexity awareness built into them, the principles that make them complexity-aware have not been articulated.
Complexity Aware Monitoring, Evaluation and Learning (CAMEL) is a new way of thinking about the principles necessary to embed complexity awareness into program monitoring or evaluation. Rather than a standalone approach or method, CAMEL is about how practitioners collect data, identify attribution, and understand outcomes. (Table 1) “The principles can be applied to a variety of approaches,” says Dr. Yordanos Molla. The key is knowing when CAMEL might be useful.
Molla explained the application of CAMEL on the Abt-led USAID Local Health System Sustainability (LHSS) Project. LHSS led a COVID-19 learning activity that explicitly incorporated diverse perspectives, emergent outcomes, and uncertain causal links to explore the effects of the COVID-19 response on health systems. Using a combination of methods to understand which change story was important to each stakeholder, LHSS discovered that countries used COVID-19 funding to improve surveillance systems and strengthen pharmaceutical warehousing, vaccine manufacturing, and laboratory management and diagnostic capabilities.
Applying CAMEL principles doesn’t have to be hard.
Applying CAMEL principles requires openness to embedding the questions in Table 1 into monitoring and evaluation (M&E) tools, approaches, and processes.
Dr. Chinwoke Isiguzo gave an example of how Palladium integrated CAMEL principles into monitoring systems. One critical principle: including diverse perspectives into routine monitoring to improve understanding of stockouts in Nigeria. The project reached out to women who had received family planning (FP) counseling, but not services. That enabled Palladium to bring the service users’ perspective to the logistic management coordination unit at the Ministry of Health and the partner in charge of the distribution so they could develop solutions. Eventually about 80% percent of the women got FP services.
CAMEL principles encourage us to think differently about outcomes.
The principles are most helpful when the chain of events that led to our desired outcome are unclear, and traditional methods can’t capture the factors that lead to the outcomes. Unresolved issues may include attribution of inputs to outcomes, the effects of context, and stakeholder motivations. Instead of a traditional approach focused on studying if an intervention produced a desired outcome, CAMEL encourages us to start with outcomes and work backward, using qualitative discussions to identify the causes of the outcomes.
Dr. Soumya Alva highlighted this point in her discussion of JSI’s inSupply project in Kenya, which sought to strengthen the supply chain in nomadic communities. In Kenya, the variety of funders, activities, intended outcomes, and environments made using traditional monitoring techniques infeasible. So JSI used CAMEL principles in both outcome harvesting and ripple effects mapping approaches. The project interviewed pharmacists and health providers to identify both final and intermediate outcomes and tested those outcomes among more stakeholders. That led to discovery of the contextual factors that influenced outcomes and clarified the pathways and relationships that led to achieving both intended and unintended outcomes.
CAMEL and systems thinking are inseparable.
Systems thinking focuses on interconnectedness and relationships among context, inputs, attribution, assumptions, and outcomes, while CAMEL provides ways to understand dynamic systems with multiple connections and uncertain outcomes. This feature of CAMEL makes it a set of principles that are particularly well adapted to evaluating health systems strengthening interventions.
At the same time, CAMEL does not replace more traditional monitoring and evaluation approaches that track the link between interventions and intended outcomes. Indeed, applying approaches such as randomized controlled trials, causal inference methods for non-randomized situations, and analysis of big data can help identify WHICH interventions lead to health outcomes, while CAMEL principles combined with qualitative methods can help identify WHY or HOW health outcomes improve in each context, regardless of the intervention’s intended effects. That additional information is critical for knowing if an intervention is the result of unique factors or is applicable elsewhere.
In the end, CAMEL principles aren’t just a new shiny type of harpoon we can use to catch our Moby Dick. Rather the principles are new ways of approaching the whale, analyzing how whales think and move and understanding the oceans where they live. CAMEL is less about tools to catch our white whale (such as harpoons and boats), and more about understanding why whales act the way they do. With this improved knowledge, we can design and deliver policy and programmatic changes that enhance peoples’ lives.
