Over the last few weeks and months, people at BRAC in Bangladesh and at IRC International Water and Sanitation Centre in The Netherlands have been working really, really hard to prepare for our first Monitoring and Learning workshop that will happen at the end of February. Exciting, and frankly, a bit daunting to complete the full circle of planning, implementation, monitoring and learning, and adaptation for the BRAC WASH programme that covers half of Bangladesh and seeks to provide sustainable sanitation and hygiene services to almost 55 million people.
Qualitative Information System (QIS) based on Sanitation Ladders
Preparation for this upcoming workshop started a year ago, with a joint BRAC – IRC workshop to design a Qualitative Information System (QIS). QIS is a monitoring system that allows the quantification of qualitative information, such as the quality of latrines, the use of latrines, the participation of women in management and decision making and so on. The underlying assumption is that change takes place gradually and to capture this gradual change we have developed progressive scales (‘ladders’). Each step on the ladder has a short description, called a mini-scenario, which describes the situation that signifies a particular score. The ladder below was developed to measure the quality of household latrines.
| INDICATOR: Quality of Household Latrine |
SCORE |
| IDEAL: Latrine with (1) ring and slab + (2) has functioning water seal + (3) no faeces visible in pan, slab, water seal and walls + (4) latrine has two pits |
4 |
| Latrine with (1) rings and slab + (2) has functioning water seal + (3) no faeces visible in pan, slab, water seal and walls |
3 |
| BENCHMARK: Latrine with (1) rings and slab + (2) has functioning water seal |
2 |
| Latrine with (1) rings and slab, but no or broken water seal |
1 |
| No latrine or latrine without rings and slab |
0 |
| Reason(s) why score is high/not high: |
|
In total, 15 ladders have been developed to capture the key outcomes of the BRAC WASH programme: 3 for the Village WASH committees, 6 to measure the quality of sanitation and hygiene services at the household level, 1 indicator to measure the quality of water resources in the community, 4 indicators to measure the quality of WASH in schools, and finally 1 indicator to measure the quality of rural sanitation marts. More information on the QIS can be found at the IRC website.
Since, the first workshop the QIS monitoring system has gone through a number of rounds of testing and adaptation. A mobile phone application has been developed to make it possible to collect data with a mobile phone. The BRAC team pulled off an incredible effort by interviewing more than 6,000 sampled households across the project area. Right now we are analysing the data that will be presented during our monitoring and learning workshop.
SenseMaker®
Besides QIS, we have used another innovative monitoring approach: SenseMaker®. This is an approach to narrative-based research that relies on a software platform for data analysis. The sense making methodology draws heavily on complexity thinking but in a nutshell it can be best summarised as follows:
- We learn a lot from stories that are being told by our colleagues, stories that we hear from people in the villages. Stories often tell us more than a table full of data.
- However, there is a limit to how many stories you can tell and how many stories we can listen to. Hence, a methodology has been developed to help us analyse and learn from many stories that are being told by many different people.
- SenseMaker® will help us to understand the average of many stories (the wisdom of crowds) but it will also help us to see the outliers – the first signs of a new trend – more clearly.
- You need to know a bit more about the person telling the story to understand it. Think about a broken leg example. For a doctor it refers to someone who has broken their leg; for a bowler it means that he has bowled a ball with a special kind of effect, and for a gunda (bad character) in a village it can mean that he has broken someone’s leg who did not obey him.
- For that reason, we have developed the SenseMaker® framework. It helps us to understand better who is telling the story and what the story means.
- When we analyse, we first look at what the stories mean to someone and only after that we might decide to open and read certain stories.
Collecting stories. Photo: Dick de Jong
Grasping the logic of the methodology, making it work, and analysing the data definitely proved to be a challenge for all of us and without support of an external expert we would never have managed. But after developing a sense-making framework, training the story collectors, and collecting more than 1,000 stories our efforts are paying very interesting and insightful dividends. One of the interesting – but very preliminary – findings is that: “The strengths of the Village WASH Committees (VWCs) are its wide community representation and the sense of teamwork. There is little resistance to the participation in the VWC but when it is there it comes from within the household (for women) and from the wider community (for the non-poor). However, we also found that working for the VWC comes both with a level of frustration but also with a feeling of satisfaction.“
This is the first in series of blogs on IRC’s work on scaling up sanitation and hygiene services that last.
Joep Verhagen, Manager, South Asia & Latin America Team, IRC International Water and Sanitation Centre
