In Tajikistan, UNICEF is supporting the Ministry of Health and Social Protection of Population (MoHSPP) to improve supply chain performance by strengthening the collection, sharing and use of supply chain data at all levels. This case study highlights how improvements in the current system – a hybrid of paper forms and Excel-based tools – can help to facilitate and improve data collection, data visualization, and data use, which will enable managers and healthcare workers at all levels of the immunization supply chain to make better informed operational and strategic decisions.

Situation Analysis

The Republican Centre for Immuno-Prophylaxis (RCIP), which manages the national immunization programme in Tajikistan, has had an increased emphasis on strengthening its data systems in recent years. Polio reappeared in Tajikistan in spring 2010 after a 13-year absence, raising concerns about the weakness of routine immunization services and the reliability of reported coverage in the country. RCIP wanted to understand the key managerial and programmatic issues that led to the polio outbreak1. More recently, the RCIP has sought to identify the cause of periodic stock-outs and why wastage data was not reported by some locations and the lack of complete data to use for national vaccine forecasts. The increase in government funding of vaccines in recent years has also required RCIP to report in more detail on its supply chain performance rather than just immunization coverage.

It is within this context that RCIP requested technical assistance from UNICEF in 2016 through the Gavi Targeted Country Assistance (TCA) grant, to review the gaps of the existing stock management system and current data use landscape, and present options for moving toward an electronic logistics management information system (eLMIS).

Challenges to the current system

The Effective Vaccine Management (EVM) assessment in 2015 showed improvements from the previous assessment at the Central level across 8 of the 9 criteria2, with large improvements in the vaccine arrival, stock management, and information systems criteria. These improvements were attributed primarily to younger, better educated and trained staff who used an updated manual of standard operating procedures (SOPs) that has had a particular impact in their ability to manage vaccine stocks.

However, despite the improvements at central level, there were decreases in EVM scores across most criteria at subnational levels, reflecting the inconsistent, non-standard practices at district and health facilities. The updated SOPs had not been introduced at vaccine stores, nor at the district level and immunization facilities where stock and vaccine management practices continue to be inconsistent. For example, staff do not regularly review vaccine loss/damage records, nor do they carry out physical stock counts.

There were no standardized tools or forms, which made data collection, aggregation, and reporting problematic. Health workers used non-standard, homemade books at each health facility to record vaccinations, and did not consistently report information such as adverse reactions to vaccines or vial/batch wastage. As a result, RCIP regional branches did not have complete information, which affected national forecasting.

Staff at lower levels did not have adequate guidelines and procedures for temperature monitoring or stock management. At the same time, managers did not provide consistent feedback to staff on the specific actions needed to improve individual and system performance.

Data for Management intervention

In May 2016, UNICEF conducted an initial review of current data use through field visits and discussions with RCIP, the UNICEF Country Office and other partners of the Ministry of Health. The review noted that although reporting rates were high, data quality was an issue and there was a need for better data to forecast the country’s vaccine demand.

Recommendations from the visit were used to develop terms of reference for a consultant to further identify how to improve the use of data for decision-making and how to implement some of the recommendations from the expert visit.

Between November 2016 and February 2017, a UNICEF consultant worked with the UNICEF Country Office and RCIP managers at each level of the supply chain, focusing on the below key areas:

I. Revision and standardization of tools

Key forms and tools to record supply chain information at national and subnational levels were revised and standardized. These included:

  • A Stock Book used to account for the quantity of vaccines in a storeroom.

  • ‘Form #2’ used to provide logistics data, including stock levels, to RCIP and to report the number of children immunized as well as the number of children not vaccinated due to contraindications.

  • An ‘Order Form’ used to order vaccines from the higher levels and to account for the receipt of products.

  • A ‘Vial/batch wastage’ tool used to track products lost to closed vial wastage, i.e. broken, expired, exposed to temperature excursions, at each level of the supply chain; reports from each level are submitted to the next level up and ultimately to the RCIP.

  • A Child Register used to record and track the number of children receiving a vaccination.

II. Standardized coding and structure of data in existing MS Excel workbooks

Excel workbooks were already in use at RCIP, RCIP Regional Branches and a limited number of District EPI Offices, used to aggregate forms, store data, generate analytic reports, and serve as order form templates.

UNICEF and RCIP standardized the Excel workbooks and templates to include individual worksheets containing quarterly and annual aggregations, which can be used to generate Key Performance Indicators.

A standard order and number for each Region, District and Site were also implemented in the workbooks, to ensure that the aggregation and copying of data between spreadsheets is done properly. At the national level, RCIP have a master facility database with standard numbers to serve as ‘codes’ for each RCIP Regional Branch, District EPI Office, and Health Centre. Once these codes are established, the regions in a national spreadsheet, districts in a regional spreadsheet, and health centres in a district spreadsheet should always be assigned the relevant codes in a table and should always be entered in the same order. Products are also listed in the same order on relevant forms and in each relevant Excel Workbook. The master product list has a code assigned that is standardized, such as 01, 02, 03.

Standardization of the database structure at RCIP Regional Branches and District EPI Offices – such as data items captured in rows and columns – are made easier for District level data to be aggregated at the RCIP Regional Branches.

Standard operating procedures (SOPs) are used to guide staff in how to complete each of these standard Excel Workbooks, how to use the data, and how to develop data visualization, such as graphs and charts.

III. Identification of supply chain key performance indicators (KPIs)

To make it easier for managers at each level of the supply chain to use the data and subsequently encourage data use, UNICEF and RCIP identified a set of standard KPIs with agreed definitions, targets and data collection processes. This enables a common set of performance indicators across the supply chain that make it easier to identify problems and improve data-driven decisions.

Key Performance Indicators identified include:

  • On-time reporting of data: Measures the capacity to ensure commodity availability and maintain an uninterrupted supply of vaccines through routine access to logistics information for decision-making throughout the supply chain.

  • Full Stock Availability: Measures the availability of immunization products. Availability of vaccines and immunization supplies is important to reach immunization program targets.

  • Vaccine Usage Rate: Indicates the proportion of opened vials used for immunization; the remaining, unused proportion, is wasted.

  • Closed Vial Wastage Rate: Measures the potential avoidable wastage during transportation and storage.

  • Forecasted Demand Ratio: Used to validate and improve forecasting practices and assumptions, such as target population, coverage and wastage in order to increase forecasting accuracy.

  • Incidence of Stock-Outs (at central level): Measures stock availability during a specified period. In particular, tracking stock-outs at the central level is an indicator of the accuracy of forecasting and supply planning, delivery of products to the country, and the delivery of logistical data to the central level from the site/community level.

  • Cold Chain Equipment Functionality (at central level): Used to facilitate equipment servicing, maintenance or replacement.

Most KPIs are reported quarterly, which enables continuous identification and improvement of problem areas.

IV. Data visualization in the form of dashboards

Using the standard KPIs, UNICEF and RCIP created Excel Dashboard templates at RCIP and RCIP Regional Branches, where there is already good use of computers. Dashboards display the performance outcome – in the form of a statistic - for each key performance indicator. The outcome is color-coded to indicate whether performance was above target (colored green), below target but within an acceptable range (yellow), or below target and NOT within the acceptable range (in red).

figure 1

Example of dashboard showing Full Stock Availability KPI across 10 health centres.

Click the image to enlarge

Staff at Health Centres/Sites and District EPI Offices collect and then report data to RCIP Regional Branches, where data is aggregated from the large number of data points in Form #2 into a visual reflection of indicator performance. The specific indicators used in the dashboard can clearly indicate areas that are performing well or performing poorly and prompt managers to look closer at performance if indicators are below a certain target.

figure 2

Example of a dashboard workflow introduced to guide staff on how to use the dashboards for corrective action.

Click the image to enlarge

The national-level RCIP creates a dashboard of all RCIP Regional Branches to create a national picture of Key Performance Indicators on a quarterly basis. This allows directors, deputy directors, epidemiologists and data specialists to identify areas for further consideration or for improvement, and managers at the RCIP and RCIP Regional Branches can provide staff at lower levels of the supply chain with the feedback and supervision necessary to address problems immediately. Program managers can also conduct analyses into the causes of low performance to gain further insight into the reasons for such performance level. From the supervision and identification of causes, an improvement plan for actions can be implemented. Acting on this plan will improve both the indicator performance and the supply chain performance.

V. A data use manual

To ensure that staff had adequate guidance and good practice procedures for reporting, analysis, and use of data, two data use manuals were developed for RCIP and RCIP Regional Branches, and District EPI Offices and Health Centres. Both manuals guide personnel in the completion of the following supply chain management tasks:

  • Maintaining adequate supplies of vaccines
  • Recording and reporting issues and usage of vaccines
  • Ordering vaccines
  • Receiving and storing vaccines
  • Monitoring vaccine supply chain activities throughout the system

A full overview of the data use manual sections can be found in Annex 2.

VI. Inclusion of supply chain data in review meetings

UNICEF and RCIP introduced regular review meetings at regional and district level, which can allow healthcare workers at various levels to review and discuss KPIs as well as identify areas for individual and systems performance. Such meetings – held with multiple staff at higher and lower levels of the supply chain – help to ensure that staff at all levels of the supply chain understand how their data is used for data-driven solutions.

A typical supply chain review meeting agenda can include:

VII. An assessment of potential for an electronic management system

As part of the data use work, UNICEF and RCIP assessed the potential to move toward a fully electronic logistics management information system (eLMIS) from the mix of paper and Excel system in Tajikistan today. The assessment found that an electronic stock management system could replace the paper-based Form #2 reporting process, along with many other processes such as improving consumption data reporting and data usage. It also found that modules and tools could be created to improve warehouse/stock management, inventory management, cold chain assessment/management, and forecasting and quantification. The assessment noted that an eLMIS is far more comprehensive than an electronic stock management system and can include an inventory module.

However, the assessment also clarified the existing issues – the relatively high cost to establishing an eLMIS, the limited funding, as well as challenges in operating a network in such a largely mountainous country with unreliable electricity supply in some areas.

Given the relatively low number of transactions at the national RCIP and RCIP Regional Branches each month and the existing method of sharing data through emailing Excel files, the move toward an eLMIS was not recommended at this time.

Next steps in improving supply chain data use in Tajikistan

The revised SOPs and processes for use at each level of the supply chain will be finalized once a consultant is on-board, including the standardization of paper-based and MS Excel reporting formats and the automated creation of performance dashboards for each of the Key Performance Indicators using Excel. There is an expectation that piloting of the revised materials and processes will take place during the latter half of 2017 in 1-2 districts and regions, and the gradual roll-out of the approach nationwide will begin in 2018, with the criteria for selection of sites and approach to scaling-up yet to be agreed with RCIP.

Through the increasing number of younger staff working for the RCIP and their familiarity with using computers, a culture of data use is slowly developing at the higher levels of the supply chain. Over time, good practices in the use of Excel templates and Excel-produced dashboards may help expand such approaches to lower levels, particularly through the sharing of data at systematic review meetings between the same, as well as different, levels of the supply chain.

Lessons learned

While the piloting and scaling-up of improved data use has yet to commence, a number of lessons have already been identified in Tajikistan:



 

  • Assessing the current state of the entire immunization supply chain, and the identification of its strengths and weaknesses, provides a baseline from which improvements can be developed and implemented.
  • The close involvement of RCIP managers in assessing each level of the supply chain aided in the verification of data obtained, the credibility of the findings, and ownership of the recommendations by the national program.
  • Due to the large workload in assessing the strengths and weaknesses of each level of the supply chain, the revision and standardization of tools, and the streamlining of processes, the use of UNICEF consultants is often beneficial as this allows UNICEF Country Office staff to continue their ongoing support to the Government’s EPI.
  • Use of local consultants based in-country can allow for more regular follow-up of activities, especially when international consultants are not in-country.
  • Identifying and supporting Champions with foreign language skills and an interest in the data improvement issues - both in the program and the supply chain - helps maintain momentum in implementation, especially when UNICEF consultants are not in-country.
  • Assessing, revising, standardizing and streamlining data use at each level of the supply chain can often take at least two years with additional time required to train and provide further support to staff to become accustomed to the continual process of improving their own performance and that of the supply chain.
  • Improving how data is collected, reported and used is the crucial factor rather than whether it is a paper-based or a hybrid system of paper-based and Excel workbooks. The establishment of an electronic logistics management information system (eLMIS) can only bring further benefits if improvements in data collection, reporting and use have been achieved.

 

Contact your UNICEF country or regional office, or the UNICEF Supply Division in Denmark, to begin the process of improving the performance of your immunization supply chain: data4management@unicef.org.

Annex 1. Timeline for implementing improvements in EPI supply chain data use in Tajikistan

Annex 2. Sections of the data use manuals developed for central/regional and district/health facility levels

Please publish modules in offcanvas position.