Dashboard TCI Essentials Most Significant Change Technique

TCI Global Toolkit: TCI Essentials

Most Significant Change Technique

What Is It?

Using principles of learning organizations, adaptive management and implementation science as the foundation, TCI employs a mixed-methods approach, using both qualitative and quantitative data, to capture, analyze and share learnings among TCI stakeholders about the TCI model and its outcomes across different locations.

Most Significant Change (MSC) technique is the primary qualitative data collection method along with its quarterly pause & reflect exercises that TCI uses. MSC is a form of participatory monitoring and evaluation that was initially developed to evaluate social-change initiatives operating within complex community systems (Davies & Dart 2005). It is participatory because many stakeholders are involved in the process. It is a form of monitoring because it occurs throughout the project life-cycle, which informs ongoing course corrections. MSC contributes to evaluation because it provides data on impact and outcomes, which can be used to help assess the performance of an initiative as a whole.

At its essence, MSC consists of four basic steps:

Collecting stories of significant change
Selecting the most significant stories
Feeding back (or sharing) selected stories with stakeholders
Using the stories (along with quantitative and other qualitative data) to improve the initiative

What Are the Benefits?

TCI has adapted MSC as its primary qualitative data collection approach for several reasons.

The method is known to make sense of complex program impacts in dynamic contexts, such as the ones in which TCI operates. It also has been shown to capture differences in outcomes across sites and time. This is especially important for TCI since it currently works in four different regions, with the potential to expand to additional areas in the future.
Since MSC is meant to be used on an ongoing basis, the method helps support adaptive management by tracking changes as they are emerging rather than waiting until the end of the program cycle when it may be too late to make improvements.
MSC asks project stakeholders and beneficiaries these broad and simple questions: What was the situation like before your engagement with TCI? What changes have you witnessed as a result of your engagement with TCI? What do you think was the most significant change? And, why was this significant to you? What challenges have you experienced in engaging TCI and/or implementing TCI proven approaches? By deliberately framing these questions broadly, the method ensures TCI does not overlook intangible and unexpected outcomes and it allows for different perspectives from its diverse stakeholders.

How to Implement It

TCI is most interested in understanding from its stakeholders how TCI has contributed to changes in knowledge, attitudes/mindsets and practice; political and financial commitments; health systems; and service access and quality (see TCI’s Technical Brief on the Adaptation of the Most Significant Change Technique for more information). As a result, TCI Accelerator Hub staff who were trained in MSC collect monthly MSC stories by conducting individual interviews with stakeholders who have self-selected to join TCI or are implementing the proven approaches.

Step 1: Collecting Stories of Significant Change

Between June 2018 and March 2019, TCI held two- to three-day workshops with the hubs to introduce the MSC technique and provide the team with an interview guide (Appendix B of the Technical Brief) and practical interviewing tips to review.

In Nigeria and East Africa, the hub teams interview TCI stakeholders during their regular engagements – either during coaching sessions or existing meeting platforms. In Francophone West Africa, TCI Country Coordinators submit their stories via an online form and then they are selected for further development, follow-up and validation. In India, the hub team has incorporated the technique as part of its state-level review meetings in Uttar Pradesh (UP), Madhya Pradesh (MP) and Odisha. Locally, the adaptation of the MSC technique is referred to as Bus do minute aur (“Just two more minutes”) whereby every quarter the identification of stories of change from TCI City Managers is a standing agenda item at the monthly state review meetings. This allows TCI City Managers, who serve as master coaches to city implementers on the ground, to learn from each other in a safe space, which has been particularly helpful as TCI has expanded into new cities and recruited and hired new City Managers. The approach has also helped in not only orienting the new staff but also applying learnings from more experienced City Managers so that new cities can more quickly overcome similar challenges.

Step 2: Selecting the Most Significant Stories

Stories are then selected by a smaller group of individuals at the Hub level and then all stories along with the justification for the selected ones are shared with Gates Institute for cross-hub review and selection. Each of the selected stories considered by the selection committee are read aloud by domain and then discussed and voted upon by a show of hands before moving onto the next domain of change. Currently, the domains of change are as follows.

Domain of Change	Definition	Illustrative TCI Examples
Knowledge, Attitudes/Mindsets, and Practice Maps to Sustainability Pillar 2: Capacity Strengthening	The progression from awareness of an innovation, to forming positive attitudes about the innovation, to the adoption of knowledge for decision making purposes or for application in practice and policy	Increased knowledge of the TCI model Increased knowledge of proven family planning programming approaches Increased knowledge of best practices for program implementation Shifts in mindsets of the roles of implementers and coaches Rapid adoption/adaptation of TCI tools and proven approaches in service delivery, demand generation, advocacy
Political and Financial Commitments Maps to Sustainability Pillar 1: Political & Financial Commitment	Multidimensional nature of political commitment is usually captured through the level of spending on an intervention since it requires action by both the executive and legislative branches of government—implying commitment of funds and establishment of enabling policies. Statements of leaders are also commonly examined to measure political commitment.	Financial commitment of geographies Funds that are actually spent Positive statements of government officials and community and religious leaders
Systems Maps to Sustainability Pillar 3: Institutionalization	The building blocks that make health systems function well: leadership and governance, health workforce, medical products and technologies, information and research (including data demand and use); the other two WHO health systems building blocks of financing and service delivery are captured in the above and below domains, respectively.	Local ownership of programs Institutionalization of approaches Data use for decision making
Access and Quality Maps to Sustainability Pillar 4: Sustained Demand (FP Impact)	Elements that impact access to family planning services: Geographic distance Economics Administrative Awareness of services Psychosocial issues (e.g., individual attitudes, social norms) Elements that impact quality of family planning services: Choice of methods Availability of competent providers Availability of space Availability of appropriate constellation of services Appropriate client-provider interaction	Social mobilization to improve awareness of services Advocacy with religious leaders to improve individual and societal perceptions of family planning Expanding method choice Reducing commodity stock-outs

Step 3: Feeding Back Selected Stories

The selected stories are shared back with the Hubs and the Hubs also share back their selected stories with their stakeholders. This sharing may occur during regularly scheduled review meetings as well as adapted into other communication products, such as part of enewsletters, an advocacy booklet, annual review of selected stories, TCI News blog posts and forthcoming journal manuscript on the annual content analysis and meta-monitoring.

Step 4: Using the Stories to Improve Programming

Feeding stories back at regularly scheduled review meetings has fostered opportunities for diffusion of the proven approaches and learnings from their implementation to other localities, whether that includes replication in non-TCI supported health facilities or even non-TCI supported cities or fostering a healthy sense of competition among TCI-supported cities. For example, in Tanzania, TCI’s proven approaches – integrated family planning outreaches and in-reaches – have been incorporated into the workplans of non-TCI supported facilities after learning of the success of these approaches in nearby TCI-supported facilities. In Nigeria, after hearing of the impact of in-reaches in Delta State, government stakeholders in Abia and Taraba states decided to adapt the approach to their contexts. This is true for the Indore mapping and listing story in India, which defined a clear roadmap for how to replicate the proven approach in other cities. Bhopal, another TCI-supported city in India, recently adopted this strategy and found more than 50% of its slum population had been left out of previous population estimates.