ICT4D | TechChange | The Institute for Technology and Social Change

Mobile Health: How Far We’ve Come

When I first started in the field of technology for development back in around 2010, I was working specifically in global health tech, and there weren’t too many of us around. The idea of using technology in places besides the developed world was just starting to gain traction, and even then, it was limited to mostly technologists and a few health experts. Pilotitis was infecting everyone, and every project I worked on for the first few years was a pilot of some sort, with no solid plan to move beyond that stage. The focus was on finding tools and concepts that showed evidence of working, although that ended up being a more difficult task than everyone expected!

Recognizing this, the mHealth Alliance (formerly part of the United Nations Foundation) created programs to incentivize all the many organizations out there to move past the pilot stage and begin exploring how to scale their projects, and eventually other organizations followed suit. Even then, though, the health field at large was still a bit unsure about how to treat digital interventions, especially in terms of evaluation. Do you measure the success of an SMS stock monitoring tool by just counting how many times it was used, or do you count the many patients who were treated using those very drugs kept in stock? Is there even one right way to measure it?

Additionally, nobody could really agree on what terms like “scale” and “sustainability” meant for mobile health tools. It wasn’t until more recent years that the space has truly started to embrace that the definition of “scale” and “success” varies for each project, and that planning for sustainability is paramount for a mobile health product to truly make an impact.

These days, among many lessons learned, it’s widely recognized that like any technology, mobile health tools are great for targeted usages, but at the end of the day, they are just tools that cannot solve problems on their own. Even the sleekest vaccination reminder system, for instance, must be accompanied by an equally well-run vaccination program for it to really benefit a community.

The mobile health community, additionally, has expanded significantly to include a wide variety of experts and crossover roles, from doctors to designers and software developers to supply chain experts. This has contributed to an increasingly holistic view of mobile health, rather than the siloed space that it once was – which is a good thing! The more perspectives you incorporate into projects, the more likely you are to come up with a truly innovative and engaging product.

Looking Forward in Mobile Health

It’s hard to say what the next big breakthrough in mobile health will be, but we’re already starting to see a rise in the number of projects that incorporate more advanced technologies like sensors and wearables alongside mobile phones. These types of additions have the potential to reduce the amount of time and resources needed to activities that currently serve as barriers to access and utilization of health services. We’re still figuring out how these sorts of tools can be best incorporated into healthcare, even in the western world, but we’re making progress. The UNICEF Innovation team even just began testing the usage of drones to speed up the process of testing infants for HIV, which brings a whole new meaning to the concept of “mobile” health tools!

Personally, I think that the biggest changes coming ahead in the near future will not be so much in the tools themselves, but rather in how we use existing tools. With the rise of patient records and other consolidated data sources around the world, I think we’ll soon be seeing a bit more of personalized medicine in the developing world, which will help bring customized care to places where it otherwise would have been difficult to provide. Similarly, better data analytics will lead to smarter interventions – for instance, we’ll have a much better sense of exactly what types of diagnoses are prevalent in specific areas, and thus be able to better support clinicians with their precise needs. And it may sound crazy, but if you want to really look ahead, I do think we’re much closer than you’d imagine to our futuristic vision of robots supporting clinics in hard-to-reach areas. The technology for that exists; it’ll just be a matter of finding ways to make it affordable, accepted, and safe enough to use! That would open up all kinds of new doors to allowing patients with mobile phones to directly connect to their healthcare.

Regardless of what the future holds, one way that you can be prepared to participate in it is by learning more about the space at large, which is why the TechChange Mobile Phones for Public Health course is such an exciting resource! I look forward to meeting the students taking it and seeing the kind of impact they’re able to create.

About the Author
Priyanka Pathak is a technologist and digital designer whose work focuses on co-designing and co-creating technologies for social impact, especially in global health and STEM education for women. Currently, she is a Senior Designer at the Design Impact Group at Dalberg. Previously, Priyanka worked as an ICT and Innovation Specialist at the World Bank, in addition to having taught courses around design, technology, and social good at Parsons the New School and the Copenhagen Institute of Interaction Design. Priyanka holds a masters degree in informatics from Columbia University and bachelors degrees in information systems and business from the University of Texas at Austin, and is currently based in Washington, DC.

Live guest expert sessions are a key, and favorite, aspect of any of TechChange’s four-week, facilitated courses. Held 2-3 times per week, these events give our students the chance to engage with experts in the field and have active discussions. Our guest experts hail from diverse backgrounds and organizations.

Today, we had the opportunity to chat with Melissa Persaud, Director of Partnerships – North America at VOTO Mobile, who will be one of our guest experts for our upcoming mHealth: Mobile Phones for Public Health course. Melissa will be speaking during the second week of the course doing a demo of VOTO Mobile’s tools, before diving into a Q&A session with participants. Check out what she has to say about the field of mHealth!

Can you tell us about your background and experience in mHealth?

As a generalist and M4D enthusiast, I’ve been observing mHealth projects over the past 10 years. Personally, I’m excited by the possibilities technology provides in this space. From the household all the way to major hospitals, there is a role for tech. During my time with VOTO, I’ve been able to explore how pairing mobile phones and information can change and save lives.

How is mHealth incorporated to your work currently?

The beauty of VOTO is its ability to share information and promote behavior change at scale almost instantly. I get to spend my days collaborating with leading health organizations to adapt existing or design new approaches across all health needs. From finding ways to better support, train, and retain frontline and community health workers to thinking about how to disseminate critical and urgent outbreak information directly to households in real time, we are pushing the limits on what you can achieve with a basic mobile phone. I also benefit from learning from other sectors as we co-design projects across all sectors with international and local organizations around the world. For example, we’ve learned that calling Brazilians between 5-7 pm has a 4x higher response rate than 8-10 am. Sharing context-specific information like this helps us all be better development practitioners who provide even better services.

Why do you think mHealth is important / what impact can learning about the use of mobile phones for public health have on development projects?

In my mind, the health sector is one of the most diverse and complex ones out there and there is no silver bullet. With that said, tech does offer some significant enhancements to traditional health projects. You are able to reach more people directly, frequently, and cheaply. This can be incredibly useful ensuring adherence to life-saving drugs or vaccines and it was nearly impossible 15 years ago. Technology also allows us to map the spread of diseases, better understand high need areas, and allocate resources as necessary. Doctors can treat more patients and save lives through telemedicine services. Community health workers can instantly access treatment information for new diseases or rare cases they haven’t been trained on. The list goes on and on. What an opportunity!

What are your thoughts on the future of mHealth, and where you think this field is heading?

I’m particularly interested in demand driven content, where individuals can self-diagnose or learn more about health topics through their mobile phones. Think WebMD on your basic phone. A way people can learn more about their bodies and their health on their own time and without internet connectivity. I’m not sure if this is the future of mHealth, but I will be keeping an eye on it!

We believe mHealth skills have the potential to make a huge impact in your work. Why do you think taking this class is important, and who would you recommend it to (public health professionals, field workers, etc etc)?

One of the biggest challenges in mHealth is that practitioners don’t know where to start. The endless opportunities are often overwhelming and human nature is to stick with the things we know. One thing I hope the mHealth students takeaway from this course is how to think about the problem you are trying to solve and be able to make an informed decision on which mHealth tool will be most appropriate and successful. This skill will prove to be incredibly useful as students advance in their career, in health or otherwise.

Sound interesting and/or valuable to your work? Sign up for our mHealth: Mobile Phones for Public Health class here. Next session begins March 28, 2016!

About Melissa
Melissa Persaud is Director of Partnerships – North America at VOTO Mobile, a Ghana and US-based mobile engagement social enterprise. In her role, she builds and maintains partnerships with impact-oriented organizations in order to provide better choices for more voices around the globe. Personally, Melissa has a passion for program design and implementation, mobile for development (M4D), and financial inclusion. She holds a MPA in development practice from the SIPA at Columbia University and a BA from Lafayette College. Melissa is also a Returned Peace Corps Volunteer (Cameroon ’11-’13) and is currently based in Washington, D.C.

In 2015, TechChange launched the Technology for Monitoring & Evaluation Diploma Program, which combined three TechChange courses (Tech for M&E, Tech for Data Collection and Survey Design, Tech for Data Visualization) into one comprehensive program. The program was meant to give busy working professionals a robust foundation in technology for M&E through the three core courses as well as workshops and office hours with course facilitators. Our first cohort is finishing up the program as we begin 2016, and a new session will launch on January 25.

Today we are very excited to chat with Sonja Schmidt, the Senior M&E Advisor to JSI’s AIDSFree project, who is one of the first participants to complete the Technology for Monitoring & Evaluation Diploma Program: Working Professionals Experience. She discusses her experience with the overall program, how each course influenced her work, as well as how she was able to better understand the use of ICTs in M&E.

How did you come across the Tech for M&E Diploma Program?

A colleague of mine from JSI had sent around some links for TechChange courses. When I clicked the links I noticed the Diploma Program, and thought that this would be a good option to take advantage of the three courses in order to get a wider foundation on the topic.

Have you taken online courses before? Did the program meet your expectations?

I had never taken an online course before, so this was a very new experience for me. I found it challenging in the beginning, particularly with the first course that I took, because I initially felt overwhelmed and struggled a bit with learning how to move around the platform and managing the material.

That being said, the program far exceeded my expectations. I have to compliment TechChange because, being an M&E expert, I look at most material with a critical eye, but I found that the material that was put together and all of the speakers/guest experts were stellar. I was also quite pleasantly surprised by the group dynamics present on the platform. I did not expect this from a virtual group, but in the end there were names that kept popping up, and I actually had the chance to meet someone from the course in person – I am almost said that it has ended.

Are you new to the field of M&E? If not, why did you think this would be valuable to your career?

I have many years of experience in the M&E field. Despite this fact, I realized that the concept of ICT and M&E emerged on the scene pretty suddenly – it did not really exist as an articulated concept even as recently as 3 years ago. I remember meeting someone a few years back who had created his own company around an app meant to improve data collection for surveys, and was surprised because I never thought that that would take off. Now, several years later I find it fascinating how this has become mainstream.

So, my main reason for taking the program was to learn more about this new and rapidly changing field, the intersection of technology and monitoring & evaluation, and get a better grasp of it.

How have you been able to use what you learned in the courses in your work, and how has the program overall been helpful to you?

I have definitely been able to use what I learned in the courses, and the Diploma Program, as foundations for my work. The Technology for M&E course, while a bit repetitive for me sometimes, as I’m an experienced M&E professional, still provided me with exposure to new materials as well as to other people’s perspectives and approaches. The Technology for Data Collection & Survey Design course was not as applicable to my personal work, however it did improve my capacities as an M&E advisor in terms of being able to recommend methods or software, or considerations to take into account, to in-country M&E folks who might be the ones actually designing M&E programs themselves. The Technology for Data Visualization course is the one that had the most impact on my work directly, because a big part of my work is reporting to stakeholders and presenting data. The Introduction to Excel for Data Visualization course was also extremely helpful because it is a familiar software, and Excel is something that I will always use; especially for organizations that do not have much funding, Excel is a very powerful and useful tool.

In general, I think the courses were useful in my work in that when I come across a particular issue, I can now think in a way where I ask myself how I can improve or do something better. I can then go back to the material and target specific areas and continue to use the program material as a tool for learning in my work. I am also currently working on developing a training in Tanzania on data quality, and I plan to discuss with my colleagues ways to use, for example, phones to more quickly submit data from site facilities to our central office.

Interested in the TechChange Technology for Monitoring & Evaluation Diploma Program? Get more information and apply here. Enrollment is open and on-going, but our next batch of courses begins January 25, 2016. It is still not too late to sign up and join this amazing program with participants from all corners of the globe!

About Sonja
Sonja has over 15 years of experience in international public health, with a focus on infectious diseases, including TB, HIV/AIDS and immunization programs. She has long-term country experience in Bangladesh, India, Nepal and Ethiopia and has worked for several UN organizations (UNIFEM, UNICEF, WHO) and numerous USAID-funded projects. Currently as the Senior M&E Advisor to JSI’s AIDSFree project, she oversees and coordinates the monitoring and evaluation of the project and guides country projects in M&E planning, data quality assessment, data analysis and use. Sonja has an MA in medical anthropology and an MPH with a focus on policy and management.

1. Privacy

Responsible data management is not new to development. However, with the use of technology-enabled tools for M&E, it has raised a few challenges related to the privacy of individuals. These include the growing use of biometric data for tracking and sensors to monitor daily habits. The collection of personal financial information and affiliation has also made it vital to consider data security when setting up an M&E framework. This can be addressed through data encryption, ensuring that individual data is not easily identifiable, and developing a policy that ensures responsible data practices. Furthermore, organisations need to be aware of the ethical implication of collecting data on people and the necessity to secure all the permissions and consents required. It is also important to be transparent about the methods of collection, why data is collected and how will it be used with the respective individuals. Finally, ownership has to be explicit when information is shared and a plan should be in place on what happens to data collected once a project ends. In South Africa, the Protection of Personal Information Act, 4 of 2013 also lends a relevant and interesting dynamic.

2. The end-user in mind

To select the most suitable technology-enabled tool(s), taking a human-centered design approach to the selection process will ensure that the organisation does not end up with an irrelevant or unnecessary tool. The approach starts with identifying what is desirable (one should consider project managers as well as community members, i.e. the people who will be using the tool), then viewing the solution through a feasibility and viability lens. This ensures and increases the usability of the tool as well as ensuring that no segment of the community is “ignored” as result of the selected tool, i.e. thinking of the accessibility of the tool and the training that would be required. Once identified, the tool should be piloted on one project before rolling it out.

3. Getting the right balance

Technology facilitates, but does not replace, M&E methodologies such as a well-thought out theory of change and quality M&E plan. So it may be tempting to fall into the habit of selecting or collecting data based on the easiest tool rather than what really matters to your program. Furthermore, technology can lead to over-dependence on digital data and missing the opportunity to observe and interact with communities in order to get a comprehensive picture of an intervention. To get the right balance, one must be very clear on the value the tool will add.

Although there are other factors to contemplate, the above three points offer a good guide to anyone considering the use of technology-enabled tools in their programs. With the ever-growing need to understand and measure impact, the integration of technology from delivery of services and monitoring of interventions to the evaluation of programs will continue as it offers possibilities and innovation to increasing reach, moving to scale and improving the efficiency and effectiveness of interventions.

This article was originally posted on the Tshikululu Social Investments blog. Photo courtesy of Jan Truter Creative Commons.

About Amira

Amira Elibiary is a Monitoring and Evaluation (M&E) specialist with 10 years of experience in research, grant-making and program management; over two years of experience in the corporate social investment sector for education, health and social development projects. With a keen interest and extensive experience in democracy, governance, advocacy and rule of law work. Amira holds a Master’s degree in International Affairs from American University and a BA degree in Economics.

TechChange courses are designed with busy working professionals in mind. In any of our courses you will find yourself engaging with a vast network of participants from all corners of the globe who bring with them unique experiences and perspectives.

Today, we are excited to chat with Amy Noreuil, a Technology Advisor working at USAID’s Office of Transition Initiatives. Amy has taken seven TechChange courses to date, both on-demand and facilitated. We caught-up with her to chat about her overall experience with TechChange as well as how our courses have impacted her professional life.

How did you find out about TechChange, and what caught your attention about TechChange courses, or got you interested in taking them?

I’ve been following TechChange for a number of years, but I think the first time I ever heard about it was through a TechChange-hosted happy hour where I ran into Chris Neu, Chief Operating Officer of TechChange. My curiosity was piqued because I always want to know about other initiatives going on when it comes to the use of technology for social good. I love going out and hearing about new projects that are under way to figure out how they could support our work or how we could support them. I’m a contextualist at heart – I believe the impact of technology can vary widely depending on the context – so I’m always interested in learning from the experience of others. I found the sense of community and diversity of students participating in TechChange courses to be one of the biggest assets. Everyone brings a unique perspective to the ‘classroom.’

After completing your first course with TechChange, what made you decide to enroll in more?

The first course I took with TechChange was Mapping for Social Good. After that first class, what drew me in – and what has kept me coming back to TechChange – is the people. To me, virtual learning experiences are inherently more individual experiences, but TechChange courses provide the opportunity to connect with other students and take what started as a quick chat to a more nuanced conversation. The interaction can be customized to what you want and need – a quick exchange of resources (e.g. reports, toolkits, etc.) or a deeper discussion about intended and unintended impact. The user interface is easy to navigate and caters to different learning preferences, including visual learners like me. It provides a high-level survey of topics or applications, while also giving the user the option to dig into the technical details of specific tools.

You’ve now taken seven TechChange courses – how have they impacted your career as a technology advisor for USAID’s Office of Transition Initiatives?

TechChange courses allow me to discover new tools as well as share experiences and insights with a wide range of people. I’m always looking for ways to break out of traditional silos.

The three major ways in which TechChange courses have impacted my work are:

Meeting people who are passionate about the intersection of technology, media & data
Finding tools and workshopping how they could support our partners and programs
Connecting with people who bring different perspectives, ideas and approaches

What is your advice for other students participating in TechChange courses? How can they get the most out of the experience?

Come into the course with an idea of what you want to learn. Set your intention early and be open to change. This learning objective will help you navigate course content and connect with students. The facilitators are very approachable and accessible. I also really encourage participants to meet up in-person and offline with students who live in the same geographic area. There’s no replacing that face-to-face connection. Developing a community of practice and creating an environment that facilitates learning takes time and commitment. I’m excited to see the TechChange community continue to grow and change.

Interested in learning more about TechChange courses? Check out our online course catalog here! We will be launching new sessions of several of our most popular courses in the New Year!

About Amy

Amy is the Technology Advisor at USAID Office of Transition Initiatives (OTI) where she supports staff as they decide how to integrate the use of information and communication technologies into their programs. She loves supporting co-creation spaces, leading digital literacy efforts and working closely with local staff to provide an outside perspective on the design of small grants. OTI supports U.S. foreign policy objectives by helping local partners advance peace and democracy.

In international development, we all love to talk about our successes, but we need to celebrate our failures too. And failures are exactly what we will be celebrating at Fail Fest on Thursday!

As a celebration of failures, the Fail Festival looks at failures as a mark of leadership and innovation, risk-taking and pushing the boundaries of what is possible in scaling ideas from pilots to global programs.

Last year at the Fail Fest, we presented our failures with the TechChange band. We had members across our team perform with vocals, guitar, drums, oboe, and – of course, PowerPoint. We celebrated our failures, from connectivity issues when doing online training sessions on Ebola to unanticipated challenges of moving into a new office.

TechChange at Fail Fest 2014

We are excited to share our experiences in providing interactive training for social change with all the hurdles that come with it. We had a blast celebrating our failures last year, and we hope to see you at the Fail Fest on Thursday!

Today, most of us can have Pad Thai, a craft cocktail, and a professional masseuse all arrive at our doorstep with a click of a button on our phones, but the same can’t be said about data for our projects. I can’t tell you whether the thousands of schools we paid for last year were actually built and functioning! How about an on-demand service for that data-delivery?

The on-demand economy is delivering increasingly brilliant things for our daily lives – at least in advanced economies. There are so many on-demand food delivery options that investors now see the market is beginning to bottom out with saturation. Last year, over $3.89 billion, purely of venture financing, went to on-demand startups other than Uber.

But it’s yet to penetrate how we do business. First Mile Geo wants to change that.

Insights on Demand
We call it Insights On Demand. Drop a pin anywhere in the world, place a bid, task a local to capture data on your behalf, and generate near real-time dashboards, maps, and comparative analytics. No tech team, no GIS specialists, no field managers tabulating survey results. The entire process delivered, on-demand.

How it works
The process is pretty simple. Like all of the other tools you may have seen in First Mile Geo (eg mobile, SMS, websurveys, physical sensors), all you have to do is create a form or survey then select the technology for collection –in this case ‘on demand’.

Drop a pin (or multiple), set a sample size (running a survey?), set a bid on how much you’re willing to pay, and you’ll see results shortly thereafter.

As data arrives you’ll be greeted with real-time maps, dashboards, and powerpoint or pdf executive briefing documents in your preferred language.

A future envisioned
Today, there are over 4 dozen mobile data collection apps. And that’s not even including the other ways we use phones like SMS, IVR, or mass analysis of phone use patterns. But regardless of how we use these tools, data analytics can still be time-consuming: identify the need, allocate resources, create a survey or form, train enumerators, analyze results, write-up findings, brief it, and market the successes.

The future of data analytics in development, where systems are smarter and the institutional burden is lessened, is arriving. We think data delivered on demand, through services like our affiliate partners at Findyr, will have a major role to play in realizing it.

We are excited to have Matt present a demo of First Mile Geo’s Insight on Demand in our data collection course tomorrow! Interested in learning how to implement technology for your M&E needs? Check out our courses related to Technology for Monitoring and Evaluation!

About Matt

Matt McNabb is CEO of First Mile Geo and a member of the TechChange Board of Advisors. He also serves as an Adjunct Fellow at the American Security Project and a member of the Board at Epirroi, a Beirut-based management consulting firm.

By Kevin Flanagan and Yuting Liao

A few weeks ago, my colleague Yuting Liao and I had the opportunity to attend MERL Tech—an industry conference of sorts designed to bring together M&E practitioners, researchers, technologists, and development professionals—on behalf of the Monitoring, Evaluation, and Learning (MEL) team at the National Democratic Institute (NDI).

At NDI, the MEL team is always on the lookout for innovative M&E practice that can be effectively applied to the democracy and governance sector and this event seemed like an excellent opportunity to explore the “big ideas” and partake in a larger discussion: what can information and communication technologies (ICT) offer to monitoring, evaluation, and learning work as the pressure to integrate ICTs into many aspects of development programming continues to rise.

Offering nearly thirty presentations, the event provided us ample opportunity to sit back and revel in the opinions of the experts, as well as contribute meaningfully to roundtable discussions and collaborative brainstorming activities. As such, these are the five takeaways:

1. More data does not necessarily mean more learning

ICT can make data collection easier, however, it’s crucial to ask the question: is this the data we need? “Big data” is enticing and a common mistake of the novice researcher is: let’s collect as much data as we can. But will that data answer your evaluation questions or will it simply be distracting? While collecting larger volumes of data could certainly result in unexpected observations, if data collection is not strategically tied to your evaluation questions, it does not necessarily lead to better learning. Quality is more important than quantity.

2. ICT can increase the level of risk for the subjects of the evaluation

Data hacks happen, so start by being scared. Whether we want to admit it or not, ICT implementations introduce additional risks to M&E work, particularly when it comes to privacy and data security. And yet, too often M&E practitioners don’t address the risks until after a breach happens. Worry about this in advance and create a threat model to assess assets, risks, and vulnerabilities.

3. Be a data-led organization, not just data-driven

While ICT does help improve data accuracy, organizations that embrace a “data-led” mentality will empower their users to strive to better understand data and incorporate it into their decisionmaking processes. Successful learning initiatives rely on better interpretation and analysis of data, and ICT for evaluation is useless without capable analytical and sector experts.

4. ICT can expand your sample size, but be mindful of the unexpected challenges in sample bias

When collecting data, ICTs can expand the reach of your evaluation efforts, creating opportunities to capture data beyond the traditional “beneficiaries” of a program. However, the “digital divide” may perpetuate the issue of sample bias, and your results may be valid only for those segments of the population with digital access.

5. There’s no ICT “quick-fix” to improve monitoring & evaluation

While it’s possible to achieve a high level of methodological rigor through carefully designed ICT studies, it’s not always easy to do so—often being technically complex, expensive, and time-consuming. Most importantly, effective ICT is built on sound monitoring & evaluation strategies, and incorporating ICTs into M&E requires long-term institutional commitment and evaluation capacity development.

Despite the wide breadth of content, there was a common theme: “It’s ok to reinvent the wheel, not the flat tire.” These words spoken by Susan Davis during a five-minute “lightning” presentation, struck an unexpected chord with the audience, attendees and presenters alike. Whether these are words of comfort for the tech-timid or caution for the tech-tenacious, Davis pointed us all to the indisputable fact that it’s okay to look to new technologies to address old problems in development as long as we are all aware that any new process, tool, or approach has just as much potential to fall flat as did their predecessors. The successful integration of M&E and ICT is fully reliant on sound monitoring and evaluation strategies and realistic expectations.

Kevin Flanagan is a TechChange alum from Technology for Monitoring and Evaluation course. He is a learning technologist on the Monitoring, Evaluation and Learning team at the National Democratic Institute.

Yuting Liao is senior assistant for data analysis and visualization on the Monitoring, Evaluation and Learning team at the National Democratic Institute.

The National Democratic Institute is a nonprofit, nonpartisan organization working to support and strengthen democratic institutions worldwide through citizen participation, openness and accountability in government.

Featured image: Wayan Vota Flickr

In August, along with announcing our Tech for M&E Diploma program, we kicked off a M&E Professionals Series, where we will be talking one-on-one with M&E professionals to give you the inside scoop on the industry.

For this second post in the series, we are featuring an interview that one of our alumni, Stephen Giddings conducted, with Janet Kerley, Senior Director, Monitoring and Evaluation Practice at Development and Training Services, Inc (dTS), a Virginia-based consulting organization that does considerable work with USAID.

Janet Kerley is a master evaluator and an accomplished trainer in evaluation and performance measurement. As Senior Evaluator in the Monitoring and Evaluation Practice at dTS/Palladium, she provides technical leadership for evaluations in the ME unit, provides technical direction on design and field methods, and supervises the preparation of the evaluation reports. As Chief of Evaluation, Research and Measurement for the Peace Corps, she established an impact evaluation system at Peace Corps.

Ms. Kerley was the Team Leader for Monitoring and Evaluation in the Office of the Director of Foreign Assistance, US Department of State, leading a 200-member inter-agency team to develop standard indicators for the 2007 Foreign Assistance Reform reporting tool. She worked at USAID in the Bureau for Policy and Program Coordination, CDIE and as the Monitoring and Evaluation Office in the Bureau for Africa and the Bureau for Europe and Eurasia. Prior to joining USAID, Ms. Kerley was a Senior Research Associate at Aguirre International. She has lived and worked in many countries in Latin America, Africa

S: How has technology changed the way M & E is conducted over the past decade in international development?
J: The change has been remarkable! A decade ago, most of the data gathering and analysis work was all paper-based, making it difficult, time-consuming, and costly. Especially in overseas environments, it took considerable time and effort to gather, transcribe (and often translate) and analyze the data. But today, the tech tools have made data collection and analysis more efficient and save time and money.
However, there is still a considerable “digital divide” between the much more tech savvy young people and the older professionals originally trained using SPSS (or even earlier)
technologies.

S: Does paper-based data collection still have a place in M&E today?
J: Yes — in certain circumstances paper-based data collection may be preferred.

In very rural areas where electricity may not be available, where batteries for electronic devices cannot be charged or where internet connections or mobile phone services is inconsistent or not available, paper-based data collection is still the best option.

Not everyone is comfortable with data collection using electronic devices, but they may be more open to paper-based questions.

S: What are some of the pitfalls of some of the popular tech-based data collection tools?
J: With so much tech available, it is easy to get carried away.

Some less experienced or less than fully trained data gatherers may lose sight of the fundamental questions the monitoring or evaluation is trying to get at. If evaluators lack sufficient training in sound principles of research, they may be tempted to substitute technology for sound reasoning and good judgment.

Some data collection tech tools may also have a tendency to collect too much data, some of which may be irrelevant to the task at hand. USAID, in particular, is burdened by data overload where data management systems fail to filter out data that is of little use and complicates the monitoring and evaluation practices.

S: What challenges have USAID Missions faced when integrating new technologies into their M& E functions?
J: By and large, USAID Missions have been quite open to technological improvements to M&E functions. That said, there is still a “digital divide” where younger employees (including local staff) who have grown up in the digital age are more comfortable with and more adept at using new technologies to enhance M & E. But more senior and older USAID staff seem generally open to embracing and appreciating the advantages that new technologies can bring to M & E while leaving the technical analysis and the new data gathering tools to younger techie staff. USAID staff have generally been very receptive to training in using new M & E technologies to their advantage.

S: Have new evidenced based technologies made decision making by senior USAID staff easier and more informed?
J: Most USAID Mission Directors recognize the value that good evidence on performance can bring to the achievement of program results, and the added clarity that good data and visually well-presented documentation can bring to decision making.

Photo Source: UNDP in Kigali, Rwanda

S: What are the advantages of mixed methods evaluations?
J: The most important starting point for an evaluation is doing the research required to understand what questions you want answered. Only then should you begin to look at evaluation methodologies to acquire necessary information.

When done at a proper scale, well executed quantitative data collection and analytical methods can bring statistical rigor and clarity. For example, the scale of some of the evaluations done for USAID’s food security (Feed the Future) programs has generally provided reliable data. Unfortunately, USAID Missions sometimes do not make available sufficient budget to assure that sample size for quantitative methods is sufficient to draw reliable conclusions. This is where qualitative methods can help to fill gaps.

Storytelling, an evaluation tool, is one of the most useful qualitative data collection methods. Sometimes quantitative data collection methods do not allow beneficiaries to open up and provide adequate and reliable information, but they react much more positively if they are allowed to tell a story. If you get enough good stories they can provide insights and nuances that purely quantitative methods cannot. Thus mixed method evaluations can provide more reliable evidence of performance than quantitative or qualitative methods.

S: Do you think there is a bias towards quantitative methods in international development because of a lack of free and easy to use qualitative tools?
J: Not at all. Many USAID evaluations make good use of qualitative methodologies. A
decade ago, there was an overuse of “the windshield wiper” approach (an evaluation that is not given time to do adequate field work and they report what they observe “through the windshield.”) to evaluations but more recently qualitative methodologies have become more sophisticated and reliable and can provide a lot of extremely useful information for decision makers.

S: What questions should we be asking to select the best technology for M & E?
J: Evaluation planning should begin with framing the research questions — what is it that we need to learn? The preferred technological solution should be one that can best answer the research questions and must also take into account cultural sensibilities. It is very crucial that technology be viewed as a tool, and not as a substitute, for knowing the basic principles of research.

Stephen Giddings, a TechChange alum, has served for 25 years as a Foreign Service Officer with the USAID, retiring in late 2005. For most of his USAID career, he specialized in managing housing and urban development programs, serving in USAID offices in Panama, Kenya, Cote d’Ivoire, Russia and Rwanda, as well as Washington, D.C. During his last four years with USAID he was the Chief of the Policy Division for USAID’s Africa Bureau.

For the past ten years Mr. Giddings has been an independent consultant providing assistance to the Development Assistance Committee (DAC) of the OECD, and consulted with USAID, the International Real Property Foundation (IRPF), among other international development organizations. He serves on the Development Issues Committee of the USAID Alumni Association and is Co-Chair of the Africa Work Group in the Society for International Development’s Washington, D.C. Chapter (SID-Washington). Prior to his USAID career, Mr. Giddings managed low-income housing development programs at the U.S. Department of Housing and Urban Development and was Director of Planning and Development at the Boston Housing Authority. Mr. Giddings received a BA in political science from Wesleyan University and an MPA degree from the Maxwell School of Citizenship and Public Affairs at Syracuse University.

——-

Hope you enjoyed our second installment of our M&E Professionals Series! Don’t forget to follow our blog for the next post in the series!

Interested in engaging in similar conversations with M&E professionals like Stephen and Janet? Join us in our upcoming course TC211: Technology for Data Collection and Survey Design that starts on October 19. If you want the whole package, you can join our second session of our Tech for M&E Diploma program.

3D printers make creating new prosthetic limbs look easy. Smart systems enable farmers to perfectly plant, fertilize, water and harvest their fields. Innovative analytical tools allow governments, NGOs, and businesses to see trends like never before, and cloud computing technologies allow the terabytes of information created daily to be shared from partner to partner across the globe. Worldwide, Information and Communications Technology (ICT) increases output and productivity.

If utilized effectively, these technologies will build the capacity necessary to achieve the United Nations’ Sustainable Development Goals (SDGs) for 2016-2030, lifting millions out of extreme poverty as we move toward a healthier, brighter, global future. The SDGs expand upon the foundation laid by the 2000-2015 Millennium Development Goals (MDGs) by taking a more holistic approach to development issues and approaching economic, social, and environmental development as pieces of the same puzzle.

The SDG ICT Playbook guides organizations in the development sector as they leverage the power of ICT to achieve these goals, providing the context for:

Governments to build new, innovative, and sustainable ways to connect their populations to technology, thus enabling improved connection with their citizens, making processes more transparent, democratic and efficient, and improving the accessibility of government services.

NGOs to utilize this new suite of tools to conduct better research, plan more effective initiatives, and analyze their impact.

Entrepreneurs to enter into emerging markets with innovative products in an efficient, cost-effective manner that supports sustainable development.

In our work toward the SDGs, all actors should support policies, within organizations and on a national and international level, that make technology more accessible to the public.

We must create cross-sector partnerships to build the infrastructure that makes ICT possible and use those partnerships to enhance the efficacy of ICT solutions. From businesses, to governments, to organizations focused on agriculture, health, education, WASH & power, disaster relief, and environmental protection, we all stand to gain from it wouthe increased use and availability of ICT.

Acknowledging that organizations within the ICT field are situated to lead the charge on technology’s accessibility, the SDG ICT Playbook was spearheaded by a partnership between NetHope, Catholic Relief Services, Intel, Microsoft, CDW, and TechChange. While we all occupy a diverse array of organizations, we believe that our institutional differences are what give us, as a group, the holistic view that technology needs to be made accessible from a variety of perspectives, in order for it to be accessed by a variety of potential users.

Check out NetHope’s press release and blog post about the playbook.