International Small Grants Programme: review

Chapter 3: Performance of the Small Grants Programme

What has the Programme funded?

The Programme has made annual five calls for applications in 2014 to 2019, and a 2020 call which is funding one-year capacity building and feasibility studies only^[3]. The budget for each round in years 2014-2109 was £500,000. A reduced budget of £300,00 was given for the 2020 call.

Number of applications

Originally the Programme had anticipated around 50 applications per call^[4]. The actual number of applications has been considerably lower with an average of 25 applications per call. The total number of applications over all six calls has been 175 of which 150 were eligible applications.

Table 1: Number of applications to the Programme in each call

Year	Feasibility	Capacity Building	Project Grants	Total no. per year	No. of eligible applications
2014	6	2	21	29	20
2015	6	2	14	22	19
2016	5	3	21	29	26
2017	3	2	17	22	20
2018	3	5	10	18	14
2019	6	8	15	29	25
2020	10	16	*N/A	26	26
Totals	39	38	98	175	150

*The 2020 round was a limited call for Feasibility and Capacity Building grants only.

Number of awards

The original proposal for the Programme had anticipated that it would fund around 20 project grants per year (at approximately £20,000 each) and around 10 feasibility/capacity building grants (at approximately £10,000 each).

The Programme has made 80 awards in total in response to the six funding calls (2014-2019), which breaks down to:

• 42 Project grants (average 7 awards per call)
• 20 Feasibility grants
• 18 Capacity Building grants

Spend per year

The budget for the Programme was £500,000 p.a. in each of the 2014-2019 rounds. The budget has not been fully spent in any year. Table 2 presents the number of awards made in each call and the total spend in each year. Note the decisions on the 2020 call are pending at time of writing and are not included.

Table 2: Number of awards and spend in each call 2014-2019

	Awards
Year	Feasibility Grants	Capacity Building	Project Grants	Total No. of awards	Total spend per funding call
2014	3	1	7	11	£431,279
2015	5	2	8	15	£459,201
2016	4	2	8	14	£476,924
2017	1	1	8	10	£479,117
2018	2	5	5	12	£338,810
2019	5	7	6	18	£486,525
Total #	20	18	42	80
Total £	£187,362	£164,666	£2,301,828		£2,653,856

Number of award-holders

The 80 awards were made to 43 organisations. Several organisations have made multiple applications and have received several awards. Of the 43 award holders, 25 organisations have secured one grant and 18 have secured 2 or more awards. (See Table 3 Appendix 1 for full breakdown).

Profile of award-holders

There is no record of the turnover of applicants/awardees at the point of application. Using OSCR data, we have estimated the profile of successful applicants. (See Appendix 1 for method and limitations of data.) This indicates that more than half of all grant holders are either very small or could be termed as micro-organisations. Of the 43 organisations:

• 16 (37%) had an income of less than £25,000.
• 8 (19%) had an income of less that £50,000
• 8 had an income of £50,000 to £100,000,
• 6 had income between £100,000 and 200,000,
• 1 had an income of over £200,000.

Across the 6 calls there were 18 applications from diaspora-led organisations. Note that this is number of applications, not organisations and some applicants made several applications. From 18 applications, 6 awards were made over the period.

Spend profile in partner countries

Table 3 shows the number of awards, the category of award and the spend in each country over the period 2014-2019. Just under half (49%) of all spend has been awarded to organisations that focus on Malawi, followed by 14% on Tanzania, 13% on Zambia, 11% on India, and 3% each on Bangladesh and on Rwanda. The spend in each of the remaining countries represents less than 0.5% of the total Programme spend.

Table 3: Number of awards and spend in each country

	Feasibility grants	Capacity Building grants	Project grants	No. of Awards	Total Spend £
Malawi	4	5	22	31	1,292,695
Zambia	2	4	6	12	375,084
Tanzania	6	4	5	15	349,155
India	1		5	6	292,385
Bangladesh	2		1	3	79,940
Rwanda	1		1	2	69,620
Zambia/Tanzania	1		1	2	68,200
Malawi/Zambia			1	1	59,800
Scotland/Tanzania		2		2	17,697
DRC	1			1	10,000
Myanmar/Scotland		1		1	10,000
Mozambique		1		1	9,577
Kenya	1			1	9,517
Pakistan	1			1	7,386
*Scotland		1		1	2,800
Total	20	18	42	80	2,653,856

*Capacity building grants do not need a partner and funding can be spent on building capacity of the Scottish based organisation

Fit with priority themes

The assessment process ensured that all successful applicants addressed the priority themes. Table 4 shows the themes that award holders addressed over the 6 funding rounds from 2014 to 2019. The learning reports highlight that many applicants identified that their work addressed more than one theme, so the number of themes (96) is greater than the number of awards (80).

Table 4: Priority themes addressed by the Programme

Theme: Number of awards focusing on this theme

• Education: 22
• Health: 31
• Sustainable economic development: 27
• Civic Governance/society: 2
• Food security: 6
• Renewable energy: 5
• Climate change: 2
• Water: 1

Has the Programme delivered on its strategic purpose?

The assessment process ensured that all funded projects could demonstrate 'fit' with the specific Programme objectives. Any application that did not meet the Programme objectives was not recommended for funding.

However, this section considers the extent to which the Programme has achieved its wider purpose and dual strategic aims of:

• growing scale and capacity of the international development sector in Scotland
• enhancing Scotland's contribution to the global fight against poverty (development impact).

There are however two challenges in assessing the achievement of the strategic objectives:

1. The objectives are not specific and are not measurable. The Programme has multiple objectives relating to building the scale and capacity of the sector. The objectives lack clarity and there is a degree of overlap and duplication between objectives. There are no measures of success, and no indicators were developed to measure or demonstrate the achievement of the objectives.

2. There is limited data. The Programme Managers have collected data on the performance of the Programme, but in the absence of clear indicators or measures of success, there is limited data to assess the extent to which the Programme has achieved the specific objectives. The Programme does not track organisations or outcomes beyond the life of the grant, so where data does exist it is limited to the timeframe of the grant.

What evidence do we have to assess achievement of objectives?

Table 5 presents a list of the multiple objectives of the Programme and identifies the sources of evidence to demonstrate the achievement of these objectives.

Table 5 demonstrates that there is limited hard evidence on which to assess the achievement of individual objectives. In order to provide some assessment of the extent to which Programme has achieved what it set out do, we have analysed evidence from a range of sources to consider the two key aims of the Programme:

• Has the Small Grants Programme built the capacity of the sector?
• Has the Small Grants Programme delivered development impact?

The evidence for this analysis has been drawn from interviews with 15 applicant/award holders, interviews with other stakeholders and review of Programme Management information.

Table 5: Assessment of the available evidence against each objective

Aim	Sources of evidence
The stated aim of the Programme was: "to build capacity and upscale small international development organisations so that they can go on to bid for funding through the International Development Fund ("IDF") and from other funders.	Numbers of organisations that applied / secured funds through IDF and other Scottish Government funds. Evidence from interviews (NB. sample only)
Objectives
1) to enhance Scotland's contribution to the global fight against poverty through activity which is clearly designed to support the achievement of the MDGs /SDGs and economic growth in developing countries	No evidence
2) build capacity by supporting the growth of small Scottish international development organisations	Anecdotal evidence from interviews
3) develop a pool of stronger Scottish headquartered organisations that could in the future bid effectively for funding through our International Development Fund (IDF)	Limited evidence. Numbers of organisations that have applied/been awarded funds through IDF
4) engage a larger number of diverse smaller organisations – either small NGOs or community driven projects – to encourage further innovation	This objective is lacks clarity and is not measurable.
5) over time, upscale the operations of small organisations through project, feasibility, and capacity building grants	Evidence from programme management reports
6) develop a wider pool of Scottish expertise and experience by increasing the number of Scottish organisations able to engage in the Scottish Government's International Development work	Limited evidence. Number of organisations securing funding from IDF.
7) to develop understanding and learning on the effectiveness of the proposed approach to international development.	Evidence contained in Learning reports.
8) to facilitate south-south exchange, bids that additionally involve one or more other partner countries in that region will be considered as long as the main country for the block grant is identified and the rationale and benefits for inclusion of the additional countries are clearly stated.	No evidence

Has the Programme built the capacity of the sector?

A specific aim of the Programme was: "to build capacity and upscale small international development organisations so that they can go on to bid for funding through the International Development Fund ("IDF") and from other funders." This was assessed by reviewing:

1. the number of award holders that went on to secure additional funding, and

2. evidence of increased capacity of award holders.

1) Number of award holders securing additional funding

Data held by Corra Foundation allowed us to track those organisations from the Programme that went on to apply and secure funding through other Scottish Government Funds.

Table 6: Number of organisations who went to access larger funds

	Number of award holder that applied	Number that secured funding
Climate Justice Innovation Fund (CJIF)	4	2 with one holding two CJIF awards
Main Grants Programme	4	2

The data also shows that another three small organisations which had applied unsuccessfully to the Programme also applied unsuccessfully to the CJIF.

The Programme does not collect data on other sources of funding secured by Programme award holders, so we do not know the level of leveraged funds. However, interviews with a sample of applicants identified that some had secured other sources of funding:

• three organisations had secured funding from the Small Charities Challenge Fund (SCCF) funded by the Department for International Development (DFID). A further four organisations had applied but been unsuccessful,
• three had secured further funding from trusts/foundations (one award in region of £100,000)
• one (feasibility award) had enabled an organisation to secure funding for a capital project from a governmental source,
• one (feasibility study grant) had resulted in development of a new enterprise which secured Energy Catalyst Funding (Innovate UK) and had created new jobs in the partner country.

We also examined data on income levels of applicant organisations in the years after their Programme award to see if there is evidence of growth in income. This exercise was carried out using income levels reported in annual returns to OSCR. While recognising the limitations of the data, the pattern shows:

• a small number of Programme holders have seen a demonstrable increase in income over time suggesting a growth trajectory,
• the majority of grant holders show an increase in income only over the period of the grant, and a stepping down of oncome levels post-grant. This highlights the fact that project-based funding often leads to sudden increases in turnover which are not sustained beyond the end of the project.
  • funders and other stakeholders commented that this was common for small organisations and evidence of the challenges for small organisations in sustaining growth.

2) Increased capacity

Although there is no measure of 'increased capacity', interviews with award holders demonstrated that applicants had increased their capacity in different ways:

• Capacity building grants had enabled small organisations to 'buy in' specific capacity building support including support for strategic planning, for monitoring and evaluation systems etc.
• Capacity building training delivered by SIDA had built knowledge and skills in a range of topics including safeguarding, report writing etc.
• The Programme processes (for example the requirement on applicants to develop project plans, M&E plans, the requirement for annual reporting etc.) has built skills within organisations.

Although some respondents had been challenged by the Programme processes many also reported that the process, and the feedback that they received from Corra on their reports had resulted in 'a more effective, planned approach to what we are doing'. As described by one interviewee:

"It's been a developmental process. We could never have seen the opportunity to do what we are doing now without the first, early stages"

(Applicant in receipt of Feasibility and two Capacity building awards)

This demonstrates that participation in the Programme has resulted in the development of skills and experience for small organisations which improved the capacity of organisations to plan and deliver effective projects.

There is one objective which is particularly difficult to assess: Objective 4: to engage a larger number of diverse smaller organisations – either small NGOs or community driven projects – to encourage further innovation. This objective lacks clarity and it is not clear exactly what evidence would indicate success. Review of programme management reports show that the Programme has supported a small number of organisations who were not previously involved in international development, but we cannot comment on whether this has enhanced the capacity of the sector or supported innovation.

Has the Programme delivered development impact?

The Programme's design did not include any ex-post evaluation in project grants or in the contract for the management of the Programme so there is no evidence either of positive development impact or of unintended negative consequences. Within the limited scope of the review the evidence suggests that the funding delivered through the Programme has delivered the planned activities and outputs. This has been supported through robust programme management processes which has held organisations to account for the delivery of activities and through six monthly reporting.

However, there was also a small number of projects in the portfolio that did not deliver the anticipated results or lacked the scale or timelines to tackle the issue they were seeking to address. While 'failure to deliver' is not an issue that is specific to small grant holders or small organisations, the risk in the case of micro-organisations is high due to limited skills, experience and capacity (and sphere of influence) of very small organisations to address the challenges in the project countries.

Since 2017, the Programme management processes have encouraged a greater focus on gathering evidence of outcomes. Grant holders have been supported to develop monitoring and evaluation processes to improve reporting on outcomes. While some projects can demonstrate outcomes at the project level (during the life of the project grant) there is no mechanism to understand the longer-term impact at the project level, and therefore on the wider impact of the Programme.

However, stakeholders and award holders gave examples to demonstrate impact at the local level. A project partner (in Zambia) provided a written response which included:

"The trauma training is another component which has equipped the Health personnel again with knowledge on how to handle emergencies more especially road traffic accidents … So, after trauma training the district has seen reduction in victims dying at the point of accident due to, for example excessive bleeding or and having blocked airways."

The Malawi Scotland Partnership (MaSP) also highlighted the additionality of the Programme at the local level in Malawi. The Programme supported the development of some very small organisations - funding enabled them to reach individuals and communities in very rural areas that otherwise would not have been reached. MaSP also report that the Programme has helped to increase the connectedness of small organisations, enabling them to develop connections between communities, other organisations and local service providers which would be challenging for larger NGOs to achieve. Involvement in the Programme has also built the capacity of small organisations, as they have developed project management and implementation skills and gained experience which will enable them to apply for and manage larger grants.

Has the Small Grants Programme delivered value for money?

As previously stated, the original Programme design did not include any ex-post evaluation in project grants or in the contract for the management of the Programme so there is no evidence of development impact. In the absence of evidence of impact, and of an impact evaluation framework, it is not possible to comment on the extent to which the Programme delivers value for money.

At the Programme Management level, there is a robust approach to value for money which focuses on effectiveness in use of funds where:

• The assessment stage involves scrutiny of budgets to ensure that input costs are realistic (in relation to the proposed outputs) and offer effective use of funds.
• Programme managers hold grant organisations to account for spend and delivery of outputs through interim reporting.
• The Programme maximise the spend in the partner country by controlling the percentage of the fund which can be spent in Scotland (8%).

While there are robust processes for budget management, award holders commended the Scottish Government and the Programme Manager on flexibility in the use of funds. Award holders reported that they had been able to negotiate changes in budgets in response to changing circumstances which had enabled them to 'realise a better outcome'.

The original SMP/NIDOS proposal for the Programme argued that funding small organisations represented 'value for money' in a number of ways:

• low overheads associated with smaller organisations,
• high levels of volunteer input,
• the likelihood of sustained of impact through long-term relationships with partners

The proposal also anticipated that funding smaller organisations would support greater innovation.

While interviews with award holders confirmed the vast number of volunteer hours and high levels of expertise which volunteers brought to the projects, the Programme did not develop any mechanisms to test the validity of these assumptions and there is little research to validate the 'value for money' argument in relation to smaller organisations.

Learning from literature

Interviews and research on other small grants programmes reveal that the 'value for money' assumptions which have underpinned the Scottish Government's Small Grants Programme are widely quoted as justification for funding programmes aimed at smaller organisations. However, while commonly accepted, there is little evidence to back up these assumptions.

A recent review of DFID's partnerships with Civil Society Organisations (CSOs) by the Independent Commission for Aid Impact^[5] (ICAI) reports that there is little evidence that small UK CSOs are either more likely to achieve lasting impact than large CSOs or are more innovative.

• An assessment of DFID's Civil Society Challenge Fund (a fund that focused on small UK CSOs and that was terminated in 2015) found that the evidence did "not support the hypothesis that public funding of small NGOs through the fund can be justified by positive empirical evidence of its impact".
• A 2014 external evaluation of one of DFID's centrally managed funding streams came to a similar finding, and also found that smaller grantees "were not especially successful at innovation" as they had a lower risk appetite than larger CSOs, smaller networks, and were not as good at capturing lessons from innovation.

In relation to the 'value for money' argument, the ICAI review also highlights that fact that Programmes aimed at small organisations frequently require more investment in programme management as small organisations need support to apply and require increased levels of management throughout the delivery phase.

Research for the Baring Foundation^[6] identified that smaller organisations that are 'trying out new ideas' will have difficulty in demonstrating impact in the timescale demanded by institutional funders. The need for governmental funders to demonstrate impact therefore puts unrealistic expectations on funded organisations and the research exhorts funders to 'accept the long-term timescales needed for real change'.

'the rush for demonstrable results in the short term, using quantitative methods (not useful in some contexts) is often not realistic'.^[7]

In this context, the research questions the efficacy of government funding for small organisations and suggests that the requirement on government funders to demonstrate accountability and value of money means that private trusts and foundations have particular strengths as funders of smaller international organisations.

The research also questions the ability of governmental funders to provide the flexibility to meet the challenging circumstances in which many small international organisations are working, and the independence from politically driven agendas. The report again highlights that trusts and foundations can 'reduce bureaucracy, developing more sensitive approaches to accountability than governments and international agencies can'.