Karnataka, India:

Dashboards and reinforcing  feedback

 

Introduction

 

 

Karnataka (population 65 million) is a southern Indian state with diverse terrains – a long coastline on the Arabian Sea, a bio-diverse Western Ghats mountain range, a lush and thickly forested Malenadu region, and an increasingly arid Deccan plateau.

 

Reflecting the diverse ecology and history of the region, Karnataka contains some of the most socioeconomically developed districts of India (with respect to literacy and other human development indices), as well as several districts below the national average. Today, roughly 24 per cent of Karnataka’s population falls below the poverty line – for rural dwellers, this means earning less than US$7.50 per month.

 

 

figure 1
Map of Karnataka
Click the image to enlarge
In 2011, the maternal mortality ratio (MMR) in Karnataka stood at 178 per 100,000 live births and the infant mortality rate (IMR) was 35 per 1000 live births – ranking 19th in India. Among children that survive, 40 per cent under 3 years of age are underweight or stunted. Yet, Karnataka ranks 7th in India with respect to life expectancy at birth (67.9 years), which is just above the national average of 65.5 years. In terms of immunization coverage, in 2014 Karnataka ranked 4th in India with 80 per cent of children aged 12-23 months receiving full immunization.
In 2012, Bill & Melinda Gates Foundation funded efforts began to deploy a mobile logistics management information system (LMIS) at the health facility level and to measure supply chain performance impacts. The programme began in 30 primary health centres (PHCs) focusing initially on eight vaccines – bacillus Calmette‐Guérin (BCG), diphtheria, pertussis and tetanus (DPT), Japanese encephalitis (JE), oral polio (OPV), tetanus toxoid (TT), hepatitis B, measles, and rabies vaccines – with pentavalent vaccine introduced later.
Demand from district-level officials caused the programme to expand to 316 health facilities covering district stores (warehouses), community health centres (CHCs), mobile immunization units, hospitals, hospital departments, stockrooms, taluk health offices, at least one PHC in every district of Karnataka, and all health facilities across 2 whole districts serving 3.5 million citizens. The vast majority of these citizens reside in rural-, tribal- and Naxal insurgency-affected areas. Due to overwhelming demand from users (especially pharmacists, supervisors, and nurses at the lowest tiers of the supply chain) additional health commodity groups gradually became integrated in the LMIS reaching more than 140 items including vaccines, essential medicines, emergency medicines, diagnostic kits, surgical items, tuberculosis medicines, mental health medicines, family planning and reproductive health commodities.
figure 2
Three tiers of the district
Click the image to enlarge
The following sections describe how data and analytics were used to improve decision making1 and ultimately changed behaviours, such that the performance of the supply chain improved drastically over a span of 14 months.

1Ramanujapuram, Arun, and Akkihal, Anup, ’Improving Performance of Rural Supply Chains Using Mobile Phones: Reducing information asymmetry to improve stock availability in low-resource environments’, ACM DEV-5 ’14 Proceedings of the Fifth ACM Symposium on Computing for Development, ACM New York, 5 December 2014, pp. 11–20, accessed 18 November 2015.

Situational Analysis

Health system dysfunction is dynamically complex. The focus herein is on problems affecting vaccines and immunization programmes – ignoring the more serious deficiencies that plagues the state’s essential medicine supply chain. Immunization programmes are relatively well managed by the Department of Health and Family Welfare (DHFW). However, there were previously no quantitative supply chain benchmarks for stock-outs for any commodity group, so problems were only understood via dozens of field visits over the 14-month period to interview system users such as NGO supervisors, district health officers, district warehouse supervisors, taluk health officers and most importantly pharmacists and nurses at primary health centres (PHCs). During the course of the intervention, user behaviours and supply chain performance were also carefully monitored.

Notable issues identified:

  • PHCs were (and remain) grossly understaffed. Many pharmacists and nurses must keep stock records for multiple facilities.
  • Paper stock reports rarely travel upstream for analysis.
  • Pharmacists are expected to visit higher-level DHFW facilities (such as district cold rooms or taluk health offices) to put signatures on ‘indent’ (order) forms, even if physical delivery is offered via cold chain-equipped vehicles. Pharmacists characterize this as an unnecessary burden on an already overstretched workforce. Further, most replenishment decisions occur on-the-spot when the pharmacist travels to the supplying facility for pickup.
  • Pharmacists and nurses were largely unmotivated to follow standard operating procedures (SOPs) because supplying facilities did not pay attention to paper records anyway – except to penalize people for wastage. Moreover, there were too many records to keep – in some cases up to nine different record books for a nurse.
  • During the course of the programme’s deployment, it became clear that even with standard web dashboards (such as real-time stock reports, order reports, with visualizations), several taluk health officers and supervisors did not even log into the web dashboard. This encouraged thinking beyond traditional web-based dashboards to instead push actionable information and decision support through novel communications channels.
  • There was previously no evidence suggesting under-provision of vaccines. However, significant delay and uncertainty surrounded the launch of pentavalent in 2013. This indicated a lack of communication and coordination throughout the channel. While undersupply may not have been an issue in aggregate, ‘silo-thinking’, or a failure to share information with others, prevented preparedness downstream for a sudden influx of new vaccines.
  • The most direct problems affecting availability of stocks were forecasting failures and occasional under-supply at the state level. Staffing and data collection challenges were also important problems identified.

   

Implementation

 

Upon a lengthy context landscaping, the programme set forth to first implement a data collection system and standard logistics dashboards. Data collection requirements were deliberately simple (see Figure 3).

 

figure 3
Basic processes between facilities
Click the image to enlarge
The selected media for data collection were basic low-end mobile phones that supported Java (J2ME) and initially dashboards accessible through standard web browsers (e.g., Firefox, IE, Chrome and Safari) were used as the media for data analysis. The idea was to collect data in a way leading to rapid analysis and effective presentation to key decision makers in the supply chain. The images below provide an idea of the system originally implemented.

After the initial three months of deployment, adoption of the system by users was still only moderate (measured by number of transactions – similar to ‘reporting rate’). While some pharmacists and nurses were consistently reporting data through the platform, others ignored the SOPs, hoping the programme ended as quickly as most interventions before it. More surprisingly, most of the supervisory class of users, who are consumers of analytics – such as district health officers, taluk health officers, warehouse managers, and district-level supervisors – were not even logging into the web dashboard to view information. In fact, several attempted login exactly zero times.
figure 4

 

Mobile data collection on basic handsets
Click the image to enlarge
Because social rewarding and shaming are known to be a rudimentary, yet powerful, mechanism to elicit behaviour change within groups, a decision was made, with programme champions at the state level, to exert social pressure to encourage users to perform. Supply chain events (e.g., stock-outs, maximum overstock conditions, adjustments beyond a reasonable percentage, minimum stock conditions, unprocessed orders, late delivery, etc.) were shared publicly with all members of the supply chain network, alongside more personally sensitive behaviour-related events (e.g., user not logging in for several days, users failing to report data within reasonable time frames, excessive wastage, etc.). These events were shared via the ubiquitous medium of television (TV) using an Android-USB stick that also pulled information wirelessly from the telecom network and were streamed to the TV at the district office and at the state headquarters of an NGO managing PHC stocks for the government as part of a public-private-partnership (PPP).
figure 5
Traditional visualizations for inventory and user behaviour
Click the image to enlarge
The important thing was that the social networks were deep and interconnected such that the names being streamed via the television were known to the people watching it. When users began to learn that their failures were being posted publicly, there was a sudden rise in adoption of the new system and a subsequent improvement in stock management behaviour. This presentation of data in the form of actionable events on TV is known as ‘bulletin board’.

Using dashboards and feedback mechanisms

 

Supervisors viewing a problematic supply chain event on the bulletin board were quick to telephone the user requiring assistance. Once the problem was solved and the transaction was indicated digitally, the event would be removed from the TV. In this way, the bulletin board represented a feedback loop between pharmacists and nurses at health facilities and supervisors at supplying facilities, such as taluk health offices and district health offices.
figure 6
Bulletin board on a TV (left), with sample messages (right)
Click the image to enlarge

 

Initially the actionable events were mostly negative reflecting failures of the system. After the senior supervisors started providing feedback to the health workers, more positive events were also included (e.g., highlighting users who reported timely or users who experienced the fewest stock-outs) in the bulletin board. Augmenting the traditional visualization-heavy web dashboard – which requires a user to seek data and insights – with a push to use of only crisp, actionable events shared on a social medium (TV) resulted in a very clear reduction in stock-outs. Moreover, in a health system notorious for irresponsiveness and delays, the time to fix bottleneck conditions (e.g., time to replenish a stock-out or replenish a minimum stock condition) has also reduced by more than half.

 

Figures 7-9 indicate critical impacts to the management system and supply chain performance. In each illustration, it is clear that the pattern of behaviour and stock-outs was not sensitive to simple digitization or provision of dashboards. Rather, it was the introduction of a feedback loop creating meaningful human-to-human interactions (using the same actionable insights otherwise available in visual form) that most affected performance of the system.

 

figure 7
Adoption: Before and after introduction of feedback loop
Click the image to enlarge
figure 8
Availability (1.00 – stock-out) of eight routine vaccines before and after introduction of feedback loop
Click the image to enlarge
figure 9
Faster responsiveness (>62% reduction in resolution time) for rectifying stock-out and minimum stock conditions
Click the image to enlarge
Perhaps more important than tangible logistics efficiencies are the healthy new management behaviours2 observed towards the end of the study.

2Logistimo India Private Ltd., Vaccine Logistics in Karnataka (short film), <https://vimeo.com/89488545>.

Lessons Learned:

  • Staff awareness that data collected and reported matters – and needs to be acted upon.
  • More frequent, specific and meaningful interactions between staff at different tiers of the supply chain
  • Regular informal, yet data-driven, operations planning meetings
  • Creation of new administrative roles to drive data quality
  • Tangible justification of replenishment decisions during pharmacist pickups (both parties sharing assumptions based on mutually-visible data)
  • Preventative replenishment based on minimum stock conditions rather than reacting to stock-out emergencies

Main Menu - mobile