Shifting the course towards gauging impact

01 Oct Shifting the course towards gauging impact

Agastya embraces technology & improves internal processes

At Agastya, we are proud of the fact that we are one of the few non-profits that have an in-house impact assessment team. Initially, our Impact Assessment Team only dealt with small samples of data about the effectiveness of our programs, covering only the state of Karnataka. 

But our reach has been consistently growing and expanding over the years, with our programs currently serving more than 21 states across India. The broadening scope also meant that program-related data also increased exponentially. Our impact team now deals with a sample of more than one lakh every year. Recognizing the need for an evolved method of data collection, we moved from observational techniques and simple questionnaires to the Pi-Card app, which is a digital data capturing methodology that works similar to QR codes and captures responses by reading the cards via specially developed mobile app.

Challenge with huge data:
On scaling across India, our Impact Assessment Team encountered various challenges in terms of collecting and processing the data. It involved instructors spending many hours to type out students’ responses in excel sheet, this data then needed to be cleaned by our Impact Assessment team which was a rather time consuming process. There were attempts at decreasing cleaning time, however cleaning couldn’t be completely eliminated. Apart from these, authenticity of data could not be guaranteed and the whole process also resulted in usage and wastage of a tremendous amount of paper. This was a long, time consuming process and alternative methods like OMR sheets failed in pilot testing as it was difficult to ensure that children mark responses appropriately.

Curiosity is addictive:
In the beginning of academic year 2019-20, the senior manager of our Impact Assessment Team was in the midst of report creation for previous year. This involved inevitable tasks to be completed in a short span of time.  One night, as she was creating a report, she became curious on how to reduce the manual effort spent in data collection thereby get ready-to-analyze clean & structured data. She was reminded of the Pi-Card App developed by 22bate7 and arranged for a brief discussion with them. The discussion turned out positive, leading the Impact Assessment team to design the requirements that will bring every aspect of quantitative assessment under one digital platform. Our team designed the requirements with two fundamental guiding principles– quality and care. However, this required complete revamp of existing Pi-Card application with only the idea of data collection remaining the same. The development team willfully accepted the challenge. With the support from field team and developers, Impact Assessment Team rolled out Pi-Card in the record time of 38 working days!  This new digitized way has eliminated 90% manual data entry, 99% of manual data cleaning, and made it a much faster and efficient process. The responses children show are immediately captured in the mobile app, increasing data accuracy.  It has also reduced paper wastage, making it a more sustainable way of assessing data. 

Transition with Care:
Every beneficial change requires additional effort from all who are involved in it. Our Impact Assessment team ensured that this transition phase does not become a painful process for the employees. Our team wanted to set up the system with available data pre-loaded. The development team agreed, and the field team cooperated, providing required data in stipulated time. This teamwork resulted in configuration time of 2 minutes as opposed to 1-2 hours. After rolling out of the Pi-Card, teams spent most of their time in training the instructors on Pi-Card usage. Our team also came up with an idea of “Pi-Card helpline’ and created WhatsApp groups that helped with quick resolution of issues.