Building and Testing an M&E System with the Network

This three-part series explores what it takes to build an impact measurement system for a diverse global network of local actors. Drawing on the Flying Labs Network's experience, we look at the design choices, implementation challenges, and lessons that emerged as we worked to make impact measurement more useful, collaborative, and grounded in the realities of local teams.

This is the second post in a series on how the Flying Labs Network built its impact measurement system. In the first, we described why we chose to start not with technology but with the questions we wanted to answer. We revised our Theory of Change and Indicators Framework, recognized that a network built on relationships needed a connected relational database, and selected ActivityInfo as the platform to build it on without taking on the burden of maintaining that infrastructure ourselves.

But a design that makes sense on paper is not yet a system anyone can use. This post is about how we turned that design into a working tool, which we came to call the Flying Labs Log, by building and testing it with the Network itself.

Building with the people who will use it

The first version of the Flying Labs Log brought together the main areas we wanted to understand over time: projects, trainings, other ecosystem activities, members, partners, and supporters — all connected through shared reference data. The technical structure gave us the relationships we had been missing in spreadsheets, but we knew that alone would not make it a good system.

Before opening it to the whole Network, we invited a working group of Flying Labs from Africa, Asia-Pacific, and Latin America and the Caribbean to co-create, test, and shape it with us. Not only does this approach align with everything else we build for the Flying Labs Network through co-creation, but it is also the most direct answer to the risk we had named in the design phase. The way to keep impact measurement from becoming extractive is to build it with local teams, not for them. So their role was not simply to identify technical bugs. We wanted them to challenge the structure itself: whether the questions made sense, whether the workflows reflected how they actually worked, which information was difficult to provide, and where the system was becoming unnecessarily demanding.

The testing combined facilitated sessions with independent use. Together, we walked through the main forms, workflows, and reference tables, and we tested predefined tasks. Flying Labs then spent time exploring the system on their own before coming back into a collective discussion to compare experiences and recommendations. This gave us a very different type of feedback from what we could get by simply presenting the database and asking whether people liked it.

Independent testing was particularly useful because a system can feel intuitive when the person who designed it is sitting beside you explaining what everything means. The real test is what happens when someone opens it alone, tries to document an actual activity, and has to decide for themselves what a field means or which option applies.

That is where many of the most useful conversations started.

Simplicity became a design principle.

One of the clearest lessons from the working group was also one of the simplest: our M&E instinct was often to ask for too much.

When you are designing a system, every additional field can seem reasonable. Another classification might improve analysis. One more question might provide useful context later. From the perspective of the person analyzing the data, completeness can easily look like quality.

Flying Labs helped us look at the same system from the other side. Every additional field also means another question somebody has to understand, another piece of information they have to find, and another few minutes added to the process. Multiply that across many forms and dozens of organizations, and the burden grows very quickly.

One example stayed with us. We had initially proposed capturing information on the vulnerable population groups reached by Flying Labs initiatives. The information would have been valuable. But collecting it properly would have demanded a substantial data-collection effort from every Flying Labs, for something many were not positioned to gather reliably. The working group pushed back, and we dropped it. It was the right call. The lesson was not only that the value of a piece of information must justify the effort required to produce it, but also that we should not ask for information simply because it would be analytically useful if local teams cannot collect it reliably. More data is not better data if the quality cannot be trusted.

So we began changing the question from "Can this information be useful?" to "Is this information useful enough to justify asking every Flying Labs to provide it?"

That shift helped us remove fields, simplify workflows, reconsider what should be mandatory, and focus much more closely on the information that really mattered. It also changed how we thought about simplicity. Simplicity was not a concession to different levels of technical capacity; it was part of good system design. A technically comprehensive database is of little value if the effort required to maintain it eventually discourages people from using it.

This reinforced something we had built into the process from the start: a working group like this should never be treated simply as a group of testers. Its role is to help define the balance between enough structure to learn collectively and enough simplicity to make that structure realistic in everyday use.

From a working group to the wider Network

Once we had incorporated the first round of feedback, the next test was scale.

A system that works with a small group of engaged testers does not necessarily work the same way across an entire network. Before moving into regular use, we expanded the process and began sharing the system more broadly, using training sessions and a sandbox environment where people could explore the forms before entering real information. The rollout also introduced a focal point and backup approach, so that each Flying Labs had identifiable people responsible for engaging with the system.

This wider phase exposed a different set of needs. Once more people began interacting with the system, questions appeared that had not surfaced during the smaller co-creation process. Fields that seemed straightforward could still be interpreted differently. Some users needed more examples. Others understood the overall process but needed quick support at the exact moment they were entering information.

That made us realize guidance could not sit outside the system. We started paying much more attention to field labels, helper text, examples, FAQs, and short visual materials. The aim was not a large manual that people would have to read before opening the Log, but support available where and when it was needed. Our early experience suggested that concise, screenshot-based guidance was often more useful than long narrative documents, particularly for someone trying to solve a specific question while using the system.

This became another important lesson: usability is not only about what happens inside the database. The guidance, examples, training, and support around it are part of the system too.

Testing also means being willing to change what you built.

The value of this process was not that Flying Labs confirmed our original design. It was that they helped us change it.

Co-creation and wider testing showed us where the structure was too heavy, where terminology needed clarification, where workflows could be simplified, and what support people would need before the system could move into regular use. It also reminded us that building an M&E system for a locally led network cannot be a one-off technical exercise. Every iteration has to be tested against the realities of the people expected to maintain the information.

By the end of this phase, the system looked different from the version we had first imagined, and that was a good thing. ActivityInfo's flexibility let us refine fields, relationships, classifications, and workflows without rebuilding the underlying infrastructure each time something changed, moving gradually toward a structure that was both useful for Network-wide learning and manageable for local teams.

That is ultimately what the building and testing phase taught us: a good system is not the one that captures the most information or perfectly reflects the thinking of the M&E team. It is the one that survives contact with its users and becomes better because of it.

The system was now tested, simplified, and ready for regular use. Whether it would actually be used was a different question. Would Flying Labs make logging part of their routines? Would the information stay complete and useful over time? And could a system built to document activities eventually help us understand impact? That is where Part 3 goes.

If your network is working through similar questions about impact measurement and you find our experience useful, we are always happy to talk!