Nigeria Toolkit: Monitoring, Evaluation & Learning
Family Planning Data Review
What is it?
Data review has been defined as a process by which the accuracy, reliability and interpretation of data are verified or validated (OECD 2013). Data review involves error detection and data analysis. Error detection aims to ascertain the quality of the data and take corrective measures, while data analysis focuses on making meaning out of the available data and making informed program or policy decisions.
Data provides a solid foundation for family planning programming and policy-related decisions. The importance of informed programming (programming guided by data) cannot be overemphasized. Data can be used in a plethora of ways in FP programming. Two examples include: equitable distribution of commodities and improvements in the quality of FP service delivery. Hence, it is important to have routine FP service data that is of high quality. The National Health Management Information System (HMIS) generates health service statistics, including data on FP, on a routine basis from service delivery points or facilities. The HMIS data is often of poor quality, making it difficult for many users to trust this data (WHO 2017). As a result, it is pertinent to devise a means through which the data is reviewed on a monthly basis to improve its accuracy (i.e., what the data is intended to measure is being measured correctly) and reliability (i.e., the data is measured consistently over time) for better use.
What are the steps for implementing this approach?
The data review process involves two distinct but complementary activities – (1) error detection or data validation and (2) data analysis. The steps employed in these processes are summarized below.
Scanning and understanding the problem
The first step in a successful data review activity is to have a clear understanding of the problem to be addressed; these are data quality issues and gaps in data analysis and use at all levels. The data quality issue most likely to be tackled by data review is data inconsistency. Data inconsistency is a condition or situation whereby a data element from the same source, same reporting unit, for the same period shows different figures at different reporting levels. For example, the total number of new FP acceptors, for a health facility, in a month shows different values in the FP Daily Register and the Monthly Summary Form.
Data inconsistency can be slight or significant. When data inconsistency is slight, the difference is usually small. For example: 420 new FP acceptors for a facility in a month is reported in the register, while 418 new acceptors is reported in the summary form or the DHIS. When a data inconsistency is significant, it is typically substantial. For example, 420 new FP acceptors for a facility in a month is reported in the register, while 215 new FP acceptors is reported in the summary form or the DHIS.
Data inconsistencies in family planning occur as variances between data on the FP clinic register and the summary form or the DHIS platform. This typically means that there was a problem at the data generation level at the service delivery point and the data collation level or data entry level. Consequently, understanding the problem of data inconsistency may entail a series of diligent desk reviews. The steps for conducting a desk review include:
- Identify the health facilities to include in the review and define the period before starting the review. For instance, you can aim to include 40 facilities and review data for a period of January to June 2018 for each facility.
- Gather all the relevant documents for review. These may include the Family Planning Daily Register, the Monthly Summary sheet and data uploaded onto the DHIS.
- Compare the various data elements from the three tools/sources identified above and look for inconsistencies.
- Document the inconsistencies found for each facility, period and data element.
- Write a concise report on the desk review exercise, emphasizing the level of data inconsistencies by facility. It is advisable to visualize inconsistencies preferably using clustered bar-chart depicting data from the various sources by facility and time or period.
Data review also includes data analysis and use. The monthly/quarterly data review meeting is an avenue for health facilities to monitor in real-time the outcomes of their activities. Health facilities generate a multiplicity of data monthly from health services that they provide and report the data monthly. However, many times, they remain only data generators but are unable to use their data to appreciate the magnitude of work that they carry out and address challenges where necessary. Data review meetings provide an opportunity for health facilities to review data, share experiences, make corrections and understand when programming for a health area is working or when its failing.
Advocacy to the concerned government bodies
In system strengthening, it is essential to stimulate the interest of government officials in data and help them to efficiently utilize and leverage it from existing systems to inform decision making. Hence, all relevant government officials should be well-informed about the desk review findings, which emphasize any problems related to inconsistencies in the reported data and, as a result, their consequences. The top officers of the Department of Planning, Research and Statistics of the Ministry of Health and its agencies – the State Primary Health Care Management Board and the Hospital Management Board – among others, should be engaged during dissemination and advocacy visits. While the latter is in-charge of all primary facilities in the State, the former oversees all the secondary facilities in the State. It is important to emphasize the fact that these agencies operate independently, so there is need to carry each of them along in the process.
It is important to encourage the government to lead the process of the data review from the planning stage. This is as important as the activity itself. When government takes the driver-seat, it will increase its sense of responsibility and commitment not only to the process but also to doing something about the findings, thus ensuring action and sustainability of the process for continual data quality improvements. The Monitoring and Evaluation (M&E) Director of the State Primary Health Care Management Board is often the ideal person to lead the process. This is because, all of the M&E Officers at LGA level who are responsible for data management are under his/her supervision.
Explore existing activities to leverage
Leveraging is a key efficiency factor in any successful and sustainable program. In conducting the FP data review, it is recommended to look for existing data review activities in the State. In many States in Nigeria, there are several data review activities usually on monthly basis. These data review activities are mostly targeted at specific program data or data elements – for example, malaria and routine immunization – and therefore sponsored by implementing partners for on that specific data. The first step is to identify these activities and who is sponsoring them. Then, meet with the sponsors/organizers of each activity to discuss possible opportunities for leveraging. For example, if there is an existing data review activity on routine immunization organized and sponsored by a partner, reach out to the partner and inquire about the activity. Then, ask whether it is possible to include FP data components in the review. It is likely that they may agree to this request and allow for FP components to be included. They may also politely reject your request based on time limitations.
Useful Tip: Don’t compromise the quality of your data review by insisting that the FP data can be given a short time. Never compromise quality over cost when leveraging on other existing activities. It is fine if you do not see any suitable existing activities to effectively leverage onto. However, it should be noted that, the first choice and consideration is to leverage onto an existing activity.
In a situation whereby, there are no suitable data review activities to leverage on, plan to initiate a new review program. This might be funded either by the State or TCI, for example. Prepare an activity profile and a modest budget, this must be done with full engagement of relevant government officials. Keep the budget as modest as possible, so that the data review meetings once demonstrated can become something that the government can easily take over.
Planning meeting with all key stakeholders
Prior to the commencement of the activity, a planning meeting should be held. The planning meeting should include all key stakeholders, not only the top officials. The ideal participants may include M&E Directors from the Ministry of Health and its agencies, State MCH/RH Coordinator, LGA MCH/RH Coordinator(s), and LGA M&E Officers among others.
During the meeting, issues around data quality and inconsistencies should be discussed. The participants should be enlightened on the importance of data, the need for the data to be valid and of high quality, and consequences poor quality data.
An important output of the meeting should be a simple tool or a checklist for the data review exercise. The tool should be able address the data review needs. For instance, a data review checklist developed to address data inconsistencies may include columns for different types of data sources – FP register, summary forms, etc. – to allow for comparisons of different figures. Moreover, the tool should have a simple data analysis plan. It should also have a section for key findings and actions to be taken. The tool should be developed jointly and be kept very simple and unambiguous.
Conducting the review activity
Having come up with a good plan, it is now the time to hit the ground running. It is time to conduct the review activity. The activity can be conducted at facility level, LGA level, or at State level depending on the need and availability of funds. The checklist developed is used to review and identify data quality issues. This is followed by analyzing the verified and validated data and identifying issues and key points of actions.
Data review meetings also present an opportunity to review and address changes in data collated. The data review meeting will be held monthly or quarterly (depending on the State) at LGA level and will initially target the review of FP data among high volume sites. The review meeting will further target other important service areas to FP such as ANC, Maternal Health, Out-Patient Department, etc. The meeting will be attended by the LGA M&E focal person, FP focal person, LGA coordinator, selected health facilities and a supervisor from the State.
The meetings will be held monthly or quarterly (depending on the State) and aims to improve indices by paying attention to data and using it for decision making. Prior to the data review meeting, all LGA M&E officers send their charts to the State RME officer to review and make corrections. At the meeting, the data from all LGAs is presented for the quarter, highlighting successes and challenges faced. All selected health facilities attending the meeting would be given the opportunity to speak on challenges faced and methods that they find to work.
Reporting and feedback process
A concise report should be produced at the end of the activity. The report should include a brief background, objective(s), methodology, key findings, lessons learned, actions taken and feedback. Feedback is as important as the review itself. Any inconsistencies or data quality issues found in the course of the review should be discussed with the service providers and the record officers at the respective facilities.
Case Study: Data Review Meeting Experience in Niger State
TCI supported the Niger State Primary Health Care Development Agency (NSPHCDA) to conduct data review meetings in 9 LGAs of Niger State. Earlier in the financial year, TCI had a meeting with the NSPHCDA M&E Desk Officer and the Director of Planning, Research and Statistics to review the condition of data analysis and use in the State. The State explained that LGA M&E staff were more focused on data entry and collation from the health facilities but were not analyzing the data that they entered into DHIS. The State team stressed that while it understood the importance of analysis to catch trends, monitor progress and create discussion, it did not know how to transfer the skillset to the LGA. TCI proposed the data review meeting as a solution to this challenge and the State accepted.
A pre-activity training was conducted to update LGA M&E officers on using DHIS to download, analyze and create charts on health facility data as well as LGA level data. At the end of the training, dates were agreed upon for the data review meeting by LGA M&E staff and the State.
Prior to the data review meeting, LGA M&E officers were requested to send in the charts that they intended to present to the State M&E officer at PHCDA and TCI’s Research, Monitoring and Evaluation Technical Support Officer for final review before presentation. This intermediate level of review allowed the State to spot errors, make corrections and offer suggestions. As the data review meetings began taking place at the LGA level, they took different forms in different LGAs with the State supervising in all priority LGAs. Facility staff were able to view the data that they had collected during the quarter as a chart. This enabled the health facilities to see the indicators that their work is directly improving.
High-performing health facilities were recognized and they shared their experiences with other health facilities, encouraging peer mentorship. Data quality issues were highlighted and service delivery issues that accounted for low uptake were discussed, documented and reported for changes. A major success from the meetings was the inclusion of LGA directors who showed interest in reviewing the data alongside health facilities. Piquing the interest of stakeholders indicates that data could possibly be used in decision making at all levels – facility level, LGA, and State.
Take An Assessment and Get a Certificate
0 of 5 questions completed
You have already completed the assessment before. Hence you can not start it again.
Assessment is loading…
You must sign in or sign up to start the assessment.
You must first complete the following:
0 of 5 questions answered correctly
Time has elapsed
You have reached 0 of 0 point(s), (0)
Earned Point(s): 0 of 0, (0)
0 Essay(s) Pending (Possible Point(s): 0)
Data analysis makes meaning of available data in order to make informed programs or policy decisions.CorrectIncorrect
When a significant data inconsistency is found a desk review can be conducted to understand the problem’s origin.CorrectIncorrect
Stimulating interest of the government officials in leading the data review increases their sense of responsibility to take action to do something about the findings.CorrectIncorrect
How useful did you find the information and/or tools presented on this page? Please write your response in the box below using one of the following phrases: Very useful, Useful, Somewhat useful, Not useful.
Feel free to comment on why you made that choice.
Grading can be reviewed and adjusted.Grading can be reviewed and adjusted.
This response will be awarded full points automatically, but it can be reviewed and adjusted after submission.
How do you intend to use the information reviewed and/or tools that you accessed?Grading can be reviewed and adjusted.Grading can be reviewed and adjusted.