International Development Fund: non-communicable disease programme

Methodology

Mixed methods were used including a literature and policy review of current country specific and continent specific peer reviewed articles, country policies and international documents, and key informant interviews (KIIs). A theory of change (ToC) was developed based on the evidence. The methodology has been split into phases, to align with the deliverables and activities of the project. In reality, these phases were carried out in parallel.

Literature review

A literature review was conducted to better understand practice within the three partner countries, and throughout SSA. Searches were conducted on Embase, MEDLINE, PubMed, Global Health, Web of Science, Scopus, and EBSCOhost using search terms. Pre-identified key words were applied during the search across the databases. Further documents were identified through review of references. Peer reviewed journal articles, as well as grey literature were included in the literature review, with a specific focus on policy and evidence documents from the three partner countries, and countries with more developed NCD strategies in SSA. All evidence was accessed between September and December 2022.

Articles referring to other countries in SSA were included. Primary quantitative and qualitative studies were included in this review. These included observational studies, randomised controlled trials, non-randomised control trials, reports of interventions, and economic studies. Studies not in English or conducted outside SSA were excluded. More so, studies published before 1992 were not considered in this review. Internal reports of both the SG’s previous collaborations in partner countries and in Scotland were also included.

Policy review

A policy review was conducted to give an understanding of country priorities and progress against domestic, regional and international plans. SG strategy documents were reviewed to give an understanding of current development spend on health including of reports from previous implementing partners. Policy analysis of national NCD policies and programmes were conducted for each of the three partner countries, and where possible for other countries in SSA. In addition NCD policies and programmes from other actors active in NCDs in the partner countries were reviewed, including the private sector, local and international non-government organisations (NGOs) and UN agencies. Preidentified policy documents and reports identified by the consultants were included, including such as WHO reports and mid or final evaluations if global strategies and plans. The policy review included all NCDs including those initially considered of interest by the Scottish Government in the invitation to tender, as well as specific interventions / areas that partner countries prioritised.

Key Informant interviews

Qualitative, in-depth KIIs were conducted through online video calls from 27/10/2022 to 16/12/2022. Both consultants were present at the majority of interviews, and transcripts were taken with the consent of the KI. These transcripts were then fed into an excel spreadsheet for analysis and extraction of themes. Purposive sampling was used to form an initial list of KIs. This included members of the partner countries’ Ministries of Health, country representatives from WHO where available, and donors and health professionals from in-country NGOs. Further KIs were identified using snowballing.

An interview tool was drafted to assist these KIIs. It consisted of semi-structured questions and was developed to subtly probe informants to elicit information, opinions and experiences. A copy of this can be found in Appendix 1. The transcribed KIIs were analysed using thematic analysis conducted independently by each consultant and then cross checked. Ideas and patterns were identified in the transcripts. These were coded, and the codes were grouped into themes. A total number of 15-25 KIs were expected to be interviewed, selected for their expert knowledge of the NCD landscape within the three partner countries. In total, 48 KIIs were conducted, 16 of these with global connections were interviewed, seven from Malawi, nine from Rwanda and 16 from Zambia.

Initially focus groups were planned for discussions with colleagues in each of the partner countries. However, due to small number of actors working in NCDs in each country, and their focus on specific NCDs it was not possible to conduct robust focus groups able to generate discussion on NCDs as a whole.

A workshop with the Scottish Government International Development team was conducted online to identify common priorities and values for the programme and get feedback on proposed options for the design. The workshop used an interactive format. It was attended by five members of the team. The participants were asked to anonymously rank their priorities for the SG International Health programme, and interactive activities were carried out to generate discussion around proposed designs.

Validity

The validity of the data was ensured by using a mixed methods approach, including literature and policy review, interviews, workshops and the theory of change. Peer debriefing, through discussion with the SG’s Global South panel in February 2023 was used to further check validity. An audit trail was kept, through transcription of the interviews.

The project was designed in collaboration with the SG. Selection bias was considered a risk with the KIIs. This was limited by using purposive and snowball sampling starting with consultants’ own professional contacts and broadening these until saturation was reached. Interviewer bias was reduced by the presence of both consultants who independently transcribed conversations. Transcripts were reviewed by the consultant not conducting the interview and themes extracted for cross verification. Response bias was a relevant challenge to this process, as participants may feel there are potential gains individually or as an organisation from this project. Open questions were used, which were neutrally worded. Having prolonged engagement with participants mitigates the risk of participant bias; however, this was not always possible due to the relatively short length of the project. Funding decisions will be made using SG processes further limiting the potential for participant bias.

Limitations

The greatest limitation was that the consultants conducted this review remotely, outside the partner countries and that both consultants were non-national staff. Whilst the consultants’ network and range of contacts in the three countries, alongside their experience working in the partner countries helped mitigate this to some extent, some potential contacts or connections may have been missed due to the timeline and the way in which country contacts were identified which may have resulted in some inherent bias. In addition, there was a reliance on national colleagues in providing cultural and local knowledge.

The snowballing used by the consultants to identify KIs is time-consuming and given the short time frame available to conduct the research, does leave the potential for exclusion of local experts. This was most pronounced in Rwanda where snowballing took the most time in terms of identification of potential KIs. This was mitigated by using triangulation from multiple sources to ensure all relevant KIs were reached. In Malawi and Zambia, saturation was achieved sooner than in Rwanda.

Participants in the KII may have felt that there was a potential for gain through the interviews. To ensure there was minimal bias in the interviews, the consultants were clear that they were not involved in the final selection process, and the KIIs were for information collection only.

Initially, the consultants planned to conduct focus group discussions (FGDs), however, due to the relatively low number of organisations working in the NCD field, the wide range of areas covered under the NCD umbrella, with many KIs having specialist interests in a particular NCD, FGDs were not felt to be the most efficient or effective method of information gathering. FGDs may have offered a different dynamic and generated more varied data.

Ethics

Multiple sources for identifying stakeholders were used including SG contacts, consultants’ professional contacts, Ministry of Health (MoH) contacts, and NGO contacts. All participation in KIIs was voluntary. Participants were informed they are free to withdraw or leave at any point, without having to provide a reason and with no negative repercussions from withdrawal. Informed verbal consent was gained for all KIIs. Interviews were transcribed by the consultants during the interview. Identities of the participants have been kept confidential and any defining characteristics removed from the final report. Interviews were conducted by both consultants to reduce the risk of interviewee bias, and to allow transcripts to occur in real time. The transcripts were reviewed by the consultant who did not transcribe to avoid any interviewer bias.