Questions raised over the results of the RCT in Rusizi

Questions raised over the results of the RCT in Rusizi

Questions raised over the results of the RCT in Rusizi

The long awaited RCT results were presented at the largest Public Health Forum in the USA at  the Water Institute Water Conference in October 2016. Gates Foundation who had  funded the research, also hosted a side event at this Conference, where Ms. Sheela Sinharoy, presenting on behalf of  IPA had shocked the audience  with a negative outcome for the much awaited study on CHCs, widely believed to be an effective methodology for changing hygiene behaviour. There had been no representative of Rwanda Ministry of Health at the meeting, and the efforts of Africa AHEAD to challenge the findings were met with passive resistance from the researchers.

The results have surprised those who have first hand experience in the use of Community Health Clubs to mobilise villagers in Rwanda.  The District Local Authority  in Rusizi are annoyed that they had not been briefed by IPA before the results were made public. As the following extract shows these findings may affect the national CBEHP Programme, based as it is, on the belief that the CHCs do in fact achieve behaviour change.

Impact of community health clubs on child diarrhea, nutritional status, and water quality in western Rwanda: a cluster-randomized controlled study

Sheela S Sinharoya, Wolf-Peter Schmidtb, Ronald Wendtc,  Leodomir Mfurac, Erin Crossettd, Karen A. Grépine, William Jackd, James Habyarimanad, Jeannine Condof, Thomas Claseng

‘Among main and secondary outcomes, neither the ‘lite’ nor the classic intervention had any impact on diarrhea, HAZ/LAZ, or WHZ/WLZ among children who had been measured at baseline or children <5, <2, or <1 year old. The intervention also had no impact on water quality as measured by E.Coli CFU TTC/100mL water.

The classic intervention had a positive impact on reported adequate water treatment methods (RD=1.09; 95% CI: 1.03-1.15), improved sanitation facilities (RD=1.09, 95% CI: 1.01-1.17), and structure of sanitation facility (RD=1.07, 95% CI: 1.00-1.14). There was no impact on the remaining intermediate outcomes, including improved drinking water source; sanitary disposal of children’s feces; presence of feces in the courtyard; presence of a handwashing station with soap; exclusive breastfeeding for children <6 months; dietary diversity for children 6-23 months; household food security; or any measures of women’s empowerment. Further analysis of the relationship between adequate water treatment and the microbiological indicator of water quality found no association (β= -19.3; 95% CI: -51.0-12.4; data not shown).’

More doubt was cast on the veracity of this research when, in the next IPA presentation, the lead investigator James Habyimana, demonstrated that there apparently had been absolutely no impact on Social Capital as a result of CHC activities. This seems to call into doubt the means of measuring social capital, which was done using a ‘experimental game’  whereby participants could either keep a small amount of money or benefit from the risk of sharing it with the group. No actual data of the extent of social networking or interdependence was  presented.

In  real life there is  extensive evidence on the ground of the increase of social networks as a result of the CHCs, and how, as a result many are CHC are  forming saving groups and improving their lives  through improved levels of cooperation which is the outcome of improved trust and reciprocity (indicators of Social Capital). These community initiatives were not mentioned by the researchers. In fact the research may have been skewed entirely because as the researchers admitted, in many case it was the husband who was asked how often his wife had attended CHC sessions, and the number of sessions attended was then associated  with the indicator of change, with the assumption the more sessions attended, the more likelihood of  change! The equivalent is to ask any western husband how many times in the past six months his wife had been to the gym!  It is not very likely you will  have an accurate response! All project records such as membership cards which do show accurate attendance were dismissed as being unavailable when in fact the  membership cards  are readily available upon request.

CHCs are being successfully rolled out across the  nation in almost all of the 14,000 villages and have recently been boosted by a multi million grant from USAID to use CHCs in a further 8 Districts to role out an Integrated WASH and Nutrition Programme known as INWA.

The findings of the RCT  certainly came as a surprise to the Health Authorities working in Rusizi District, who were largely unaware of the activities of IPA in the district.  The IPA paper concluded  ‘Our results raise questions about the value of implementing this intervention at scale’ .  This is a  clear challenge to a government programme, and could have an impact on the health of millions should the CBEHP programme stall as a result of such a facile conclusion without sufficient context to explain the challenges.  In fact the Health Centers in Rusizi consider positive  changes have taken place in their villages and were full of anecdotal stories of community response. Indeed, so impressed was the Vice Secretary for the District that he has asked Africa AHEAD to remain and finish the work on the remaining 300 villages where CHCs have not yet been started. How can there be such discrepancy between two narratives. Further evaluation is clearly needed, as issue will not go unchallenged.

This breach of protocol by the IPA researchers has already raised concerns in Rwanda, where as in Africa generally,  it is common practice that research findings are shared locally  with government before  being  presented at a public forum. A workshop is now being arranged by Gates Foundation to enable IPA to finally share the results with stakeholders in Rwanda, at the end of May, one year after the research was completed and a draft paper shared with Africa AHEAD and fully six months after IPA  have been made the results public.