Tackling child poverty pathfinders: evaluability assessment

Footnotes

1. Scottish Government Poverty and Social Justice.

2. Best Start, Bright Futures - Scottish Government Tackling Child Poverty Delivery Plan, 2022 - 2026.

3. Scottish Government Poverty and Social Justice.

4. Scottish Government Tackling Child Poverty Delivery Plan, 2022-26.

5. Scottish Government, Tackling Child Poverty Delivery Plan, 2022-26.

6. Best Start Bright Futures Delivery Plan.

7. Best Start Bright Futures Delivery Plan.

8. Individuals from the Glasgow Pathfinder were unable to attend due to illness. For a full list of participants at the Programme Level workshop see Child Poverty ToC and MEL report.

9. Best Start Bright Futures, Scottish Government Tackling Child Poverty Delivery Plan.

10. Ministry of Housing, Communities & Local Government. (2021). Supporting Families 2021-22 and Beyond.

11. Office of the First Minister and Deputy First Minister. (2013). Best Practice in Addressing Child Poverty.

12. White, H., (2006). Impact evaluation: the experience of the Independent Evaluation Group of the World Bank, P.1,3,4,9.

13. This would most likely be Glasgow city and Linlathen in Dundee for the two respective Pathfinders.

14. The latest national statistics on households below average income show that in the 20 years from 2002/03-2021/22, the percentage of children in absolute or relative poverty changed on average by 1 percentage-point per year. Over that period, absolute child poverty fell by 10 percentage points and relative child poverty fell by 2 percentage points.

15. Welsh Government. (2014). Evaluation of the Welsh Child Poverty Strategy Final Report.

16. Morton, S. (2018). Evaluating Health Technology Wales.

17. Cabaj, M. (2019). Evaluating Systems Change Results.

18. Cordis Bright. (2020). Evaluating Systems Change.

19. This is consistent with Magenta Book guidance, which considers (quasi-)experimental and impact assessment (or 'theory-based') approaches as the two main impact evaluation methods.

20. White, H. & Sabarwal, S. (2014). Quasi-Experimental Design and Methods.

21. White, H. & Sabarwal, S. (2014). Quasi-Experimental Design and Methods.

22. White, H. & Sabarwal, S. (2014). Quasi-Experimental Design and Methods.

23. White, H. & Sabarwal, S. (2014). Quasi-Experimental Design and Methods.

24. Ministry of Housing, Communities & Local Government. (2019). National Evaluation of the Troubled Families Programme 2015-2020.

25. Jalan. J. & Ravallion, M. (2003). Estimating the Benefit Incidence of an Antipoverty Program by Propensity-Score Matching.

26. Ministry of Housing, Communities & Local Government. (2019). National Evaluation of the Troubled Families Programme 2015-2020.

27. HM Treasury. (2020). Magenta Book Annex A.

28. HM Treasury. (2020). Magenta Book Annex A.

29. HM Treasury. (2020). Magenta Book Annex A.

30. HM Treasury. (2020). Magenta Book Annex A.

31. Fredriksson, A. & Magalhaes de Oliveira, G. (2019). Impact Evaluation Using Difference-in-Differences.

32. HM Treasury. (2020). Magenta Book Annex A.

33. Fredriksson, A. & Magalhaes de Oliveira, G. (2019). Impact Evaluation Using Difference-in-Differences.

34. Fredriksson, A. & Magalhaes de Oliveira, G. (2019). Impact Evaluation Using Difference-in-Differences.

35. Fredriksson, A. & Magalhaes de Oliveira, G. (2019). Impact Evaluation Using Difference-in-Differences.

36. HM Treasury. (2020). Magenta Book Annex A.

37. MDRC. (2019). Using a Regression Discontinuity Design for Evaluation Studies.

38. MDRC. (2019). Using a Regression Discontinuity Design for Evaluation Studies.

39. HM Treasury. (2020). Magenta Book Annex A.

40. HM Treasury. (2020). Magenta Book Annex A.

41. HM Treasury. (2020). Magenta Book Annex A.

42. MDRC. (2019). Using a Regression Discontinuity Design for Evaluation Studies.

43. HM Treasury. (2020). Magenta Book Annex A.

44. HM Treasury. (2020). Magenta Book Annex A.

45. HM Treasury. (2020). Magenta Book Annex A.

46. BMJ. (2021). A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance.

47. BMJ. (2016). CONSORT 2010 statement: extension to randomised pilot and feasibility trials.

48. BMJ. (2010). CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials.

49. BMJ. (2010). CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials.

50. Ministry of Housing, Communities & Local Government. (2019). National Evaluation of the Troubled Families Programme 2015-2020.

51. Carolina Population Center. (2014). Livelihood Empowerment Against Poverty Program Impact Evaluation.

52. Levy, Y. et al,. (2011) A Guide for Novice Researchers on Experimental and Quasi-Experimental Studies in Information Systems Research.

53. White, H. & Sabarwal, S. Quasi-Experimental Design and Methods.

54. Australian Government. (2021). Evidence and Evaluation Support.

55. Australian Government. (2021). Evidence and Evaluation Support.

56. Australian Government. (2021). Evidence and Evaluation Support.

57. Australian Government. (2021). Evidence and Evaluation Support.

58. Department for Digital, Culture, Media & Sport. (2023). Volunteering Futures Fund evaluation: feasibility study.

59. Malmberg-Heimonen, I. and Tøge, A.G. (2022) Family Intervention Projects as Poverty-Alleviating Measures: Results from a Norwegian Cluster-Randomised Study.

60. Wang, J., Malaeb, B., Ssewamala, F., Neilands, T., and Brooks-Gunn, J. (2021). A Multifaceted Intervention with Savings Incentives to Reduce Multidimensional Child Poverty: Evidence from the Bridges Study (2012–2018) in Rural Uganda.

61. U.S. National Library of Medicine. (2022). A Randomised Trial of Healthier Wealthier Families in Sweden.

62. Theory-Based Approaches to Evaluation: Concepts and Practices - Canada.ca.

63. ibid

64. ibid

65. Morton, S. (2019).

66. Pawson, R., & Tilley, N., (1997). Realistic Evaluation

67. White, H. and Phillips, D., (2012). Addressing Attribution of Cause and Effect in Small n Impact Evaluations: Towards an Integrated Framework.

68. Lucas, H. and Longhurst, R., (2010). Evaluation: Why, for Whom and How? p.31.

69. HM Treasury. Magenta Book Annex A Analytical methods for use within evaluation.

70. HM Treasury. Magenta Book Annex A Analytical methods for use within evaluation.

71. Morton. S. (2019)

72. The Dundee Monitoring System is yet to be developed, suggestions for why this and how this might take place is included in the ToC and Monitoring Report.

73. Dundee City Council and Social Security Scotland

74. The Glasgow Helps Monitoring System has been developed. However, a few adaptations will need to be made in order for it to effectively collate all the data needed to evaluate against the Monitoring Framework. More information is included in the ToC and Monitoring Report.

75. Holistic Needs Assessments, Exit Interviews and Financial Gain estimates are carried out by Glasgow Helps staff when they engage with families, they follow a specific structure and are recorded on MS forms. An extra level of analysis is done on the Holistic Assessments to track progress against the Glasgow Helps Monitoring System

76. Simister, N. and Scholz, V. Qualitative Comparative Analysis, INTRAC

77. Simister, N. and Scholz, V. Qualitative Comparative Analysis, INTRAC

78. Morton, S. (2019)

79. Hesslgreaves, H. et al. (2021). New development: The emerging role of a 'learning partner' relationship in supporting public service reform.

80. Hesslgreaves, H. et al. (2021). New development: The emerging role of a 'learning partner' relationship in supporting public service reform.

81. Department for International Trade (2019). VfM Guidance: The 4th E Equity.

82. An additional method not discussed here is cost utility analysis. This looks at costs relative to the amount of utility produced – the latter being based on individual preferences used to value the output of the intervention. This is typically used in the health sector and relies on quality-adjusted life years (QALYs) to measure utility. Therefore, we do not consider it relevant in the context of the Pathfinders.

83. While the ultimate aim of the Pathfinders is singular (reduce child poverty), it would not be appropriate to rely on this as the sole metric of output because (i) there are significant challenges in measuring child poverty and attributing the impact of the Pathfinders, and (ii) using a single output measure would hide the other positive benefits of the Pathfinders, which in combination may also lead to reductions in poverty.

84. GHK. (2011). Local Authority Child Poverty Innovation Pilot Evaluation: Final Synthesis Report.

85. Garfinkel, I. et al. (2022). The Benefits and Costs of a Child Allowance.

86. Impact on Urban Health (2022). Investing in Children's Future: A Cost Benefit Analysis of Free School Meal Provision Expansion.

87. Ministry of Housing, Communities & Local Government. (2019). National Evaluation of the Troubled Families Programme 2015-2020.

88. Welsh Government. (2014). Evaluation of the Welsh Child Poverty Strategy Final Report.

89. Hirsch, D. Estimating the costs of child poverty (2008). Joseph Rowntree Foundation.

90. Greater Manchester Combined Authority CBA tool.

91. HACT. (2014). Measuring the Social Impact of Community Investment: A Guide to using the Wellbeing Valuation Approach.

92. Fujiwara, D. (2010). The Department for Work and Pensions Social Cost Benefit Analysis framework.

93. Scottish Government. (2022). Tackling Child Poverty Delivery Plan 2022-2026 Annex 2.

94. MPPN. Child Poverty: How to Measure and For What Purpose.

95. Welsh Government. (2014). Evaluation of the Welsh Child Poverty Strategy Final Report.

96. Scottish Government. (2022). Tackling Child Poverty Delivery Plan 2022-2026: Annex 3.

97. Scottish Government. (2022) Tackling Child Poverty Delivery Plan 2022-2026 Annex 2.

98. HM Treasury. (2020). Magenta Book.; Scottish Government. Tackling Child Poverty Delivery Plan 2022-2026 Annex 2.

99. Scottish Government. Tackling Child Poverty Delivery Plan 2022-2026 Annex 2.

100. GHK. (2010). Local Authority Child Poverty Innovation Pilot: First National Evaluation Report.

101. GHK. (2010). Local Authority Child Poverty Innovation Pilot: First National Evaluation Report.

102. GHK. (2010). Local Authority Child Poverty Innovation Pilot: First National Evaluation Report.

103. HM Treasury. (2020). Magenta Book.

104. Welsh Government. (2014). Evaluation of the Welsh Child Poverty Strategy Final Report.

105. Scottish Government. Tackling Child Poverty Delivery Plan 2022-2026 Annex 2.

106. Latham, N. (2014). Systems Change Evaluation Toolkit.

107. Welsh Government. (2018). Post-Implementation Evaluation of Part 2 of the Housing Act (Wales) 2914: Final Report.

108. Welsh Government. (2018). Post-Implementation Evaluation of Part 2 of the Housing Act (Wales) 2914: Final Report.

109. Welsh Government. (2018). Post-Implementation Evaluation of Part 2 of the Housing Act (Wales) 2914: Final Report.

110. Welsh Government. (2018). Post-Implementation Evaluation of Part 2 of the Housing Act (Wales) 2914: Final Report.

111. GHK. (2010). Local Authority Child Poverty Innovation Pilot: First National Evaluation Report.

112. GHK. (2010). Local Authority Child Poverty Innovation Pilot: First National Evaluation Report.

113. GHK. (2010). Local Authority Child Poverty Innovation Pilot: First National Evaluation Report.

114. Welsh Government. (2014). Evaluation of the Welsh Child Poverty Strategy Final Report.

115. Welsh Government. (2014). Evaluation of the Welsh Child Poverty Strategy Final Report.

116. Welsh Government. (2020). Evaluation of the Out of Work Service: Final Report.

117. Ministry of Housing, Communities & Local Government. (2019). National Evaluation of the Troubled Families Programme 2015-2020.

118. White, H. & Sabarwal, S. (2014). Quasi-Experimental Design and Methods.

119. White, H. & Sabarwal, S. (2014). Quasi-Experimental Design and Methods.

120. White, H. & Sabarwal, S. (2014). Quasi-Experimental Design and Methods.

121. Morton, S. (2018). Evaluating Health Technology Wales.

122. White, H. & Sabarwal, S. (2014). Quasi-Experimental Design and Methods.

123. White, H. & Sabarwal, S. (2014). Quasi-Experimental Design and Methods.

124. Ministry of Housing, Communities & Local Government. (2019). National Evaluation of the Troubled Families Programme 2015-2020.

125. Jalan. J. & Ravallion, M. (2003). Estimating the Benefit Incidence of an Antipoverty Program by Propensity-Score Matching.

126. Ministry of Housing, Communities & Local Government. (2019). National Evaluation of the Troubled Families Programme 2015-2020.

127. White, H. & Sabarwal, S. (2014). Quasi-Experimental Design and Methods.

128. Ministry of Housing, Communities & Local Government. (2019). National Evaluation of the Troubled Families Programme 2015-2020.

129. Ministry of Housing, Communities & Local Government. (2019). National Evaluation of the Troubled Families Programme 2015-2020.

130. Ministry of Housing, Communities & Local Government. (2019). National Evaluation of the Troubled Families Programme 2015-2020.

131. Ministry of Housing, Communities & Local Government. (2019). National Evaluation of the Troubled Families Programme 2015-2020.

132. Jalan. J. & Ravallion, M. (2003). Estimating the Benefit Incidence of an Antipoverty Program by Propensity-Score Matching.

133. Fredriksson, A. & Magalhaes de Oliveira, G. (2019). Impact Evaluation Using Difference-in-Differences.

134. Fredriksson, A. & Magalhaes de Oliveira, G. (2019). Impact Evaluation Using Difference-in-Differences.

135. Fredriksson, A. & Magalhaes de Oliveira, G. (2019). Impact Evaluation Using Difference-in-Differences.

136. Fredriksson, A. & Magalhaes de Oliveira, G. (2019). Impact Evaluation Using Difference-in-Differences.

137. Fredriksson, A. & Magalhaes de Oliveira, G. (2019). Impact Evaluation Using Difference-in-Differences.

138. Fredriksson, A. & Magalhaes de Oliveira, G. (2019). Impact Evaluation Using Difference-in-Differences.

139. Fredriksson, A. & Magalhaes de Oliveira, G. (2019). Impact Evaluation Using Difference-in-Differences.

140. Fredriksson, A. & Magalhaes de Oliveira, G. (2019). Impact Evaluation Using Difference-in-Differences.

141. Fredriksson, A. & Magalhaes de Oliveira, G. (2019). Impact Evaluation Using Difference-in-Differences.

142. Fredriksson, A. & Magalhaes de Oliveira, G. (2019). Impact Evaluation Using Difference-in-Differences.

143. Fredriksson, A. & Magalhaes de Oliveira, G. (2019). Impact Evaluation Using Difference-in-Differences.

144. Sanders, M. Trial Evaluation Protocol: Strengthening Families, Protecting Children- Family Safeguarding Model- Difference-in-differences Analysis.

145. Sanders, M. Trial Evaluation Protocol: Strengthening Families, Protecting Children- Family Safeguarding Model- Difference-in-differences Analysis.

146. Sanders, M. Trial Evaluation Protocol: Strengthening Families, Protecting Children- Family Safeguarding Model- Difference-in-differences Analysis.

147. Sanders, M. Trial Evaluation Protocol: Strengthening Families, Protecting Children- Family Safeguarding Model- Difference-in-differences Analysis.

148. Carolina Population Center. (2014). Livelihood Empowerment Against Poverty Program Impact Evaluation.

149. Carolina Population Center. (2014). Livelihood Empowerment Against Poverty Program Impact Evaluation.

150. Carolina Population Center. (2014). Livelihood Empowerment Against Poverty Program Impact Evaluation.

151. Carolina Population Center. (2014). Livelihood Empowerment Against Poverty Program Impact Evaluation.

152. Carolina Population Center. (2014). Livelihood Empowerment Against Poverty Program Impact Evaluation.

153. Carolina Population Center. (2014). Livelihood Empowerment Against Poverty Program Impact Evaluation.

154. MDRC. (2019). Using a Regression Discontinuity Design for Evaluation Studies.

155. MDRC. (2019). Using a Regression Discontinuity Design for Evaluation Studies.

156. MDRC. (2019). Using a Regression Discontinuity Design for Evaluation Studies.

157. Keele, L. & Titiunik, R. (2015). Geographic Boundaries as Regression Discontinuities.

158. Better Evaluation. Contribution Analysis.

159. Better Evaluation. Contribution Analysis.

160. Connolly, J. (2016). Contribution Analysis as an Approach to Enable Public Managers to Demonstrate Public Value: The Scottish Context.

161. Connolly, J. (2016). Contribution Analysis as an Approach to Enable Public Managers to Demonstrate Public Value: The Scottish Context.

162. Scottish Government. Guide 6: Contribution Analysis.

163. Scottish Government. Guide 6: Contribution Analysis.

164. Better Evaluation. Contribution Analysis.

165. Connolly, J. (2016). Contribution Analysis as an Approach to Enable Public Managers to Demonstrate Public Value: The Scottish Context.

166. Ghate, D. (2022). Theory of Change for Making Children's Rights Real in Scotland: Rapid Review of the Change Process no.2.

167. Ghate, D. (2022). Theory of Change for Making Children's Rights Real in Scotland: Rapid Review of the Change Process no.2.

168. Ghate, D. (2022). Theory of Change for Making Children's Rights Real in Scotland: Rapid Review of the Change Process no.2.

169. Hargreaves, M. (2010). Evaluating System Change: A Planning Guide. Princeton, NJ: Mathematica Policy Research.

170. Cordis Bright (2020). Evaluating Systems Change: Literature Review.

171. Ibid. (Cordis B)

172. Abercrombie, Harries, and Wharton (2015). Systems Change: A guide to what it is and how to do it.

173. Cordis B (2020).

174. Abercrombie, Harries, and Wharton (2015).

175. Cordis Bright (2020).

176. Cabaj, M. (2019). Evaluating Systems Change Results.

177. Latham, N. (2014). Systems Change Evaluation Toolkit.

178. Cabaj, M. (2019). Evaluating Systems Change Results.

179. Cabaj, M. (2019). Evaluating Systems Change Results.

180. Cordis Bright. (2020). Evaluating Systems Change.

181. Cordis Bright. (2020). Evaluating Systems Change.

182. Cordis Bright. (2020). Evaluating Systems Change.

183. Cordis Bright. (2020). Evaluating Systems Change.

184. CFE Research. (2022). Creating Systems Change: Evaluating the Contribution of the Fulfilling Lives Programme.

185. GHK. (2011). Local Authority Child Poverty Innovation Pilot Evaluation: Final Synthesis Report.

186. Welsh Government. (2014). Evaluation of the Welsh Child Poverty Strategy Final Report.

187. Ministry of Housing, Communities & Local Government. (2019). National Evaluation of the Troubled Families Programme 2015-2020.

188. Scottish Government. (2022). Tackling Child Poverty Delivery Plan 2022-2026 Annex 2.

189. Scottish Government. (2022). Tackling Child Poverty Delivery Plan 2022-2026 Annex 2.

190. Welsh Government. (2014). Evaluation of the Welsh Child Poverty Strategy Final Report.

191. Software such as OutNav, developed by Matter of Focus, for example, can help teams to collate relevant and useful data against outcomes and indicators, allowing the team to regularly reflect on delivery, and adapt and respond.

192. Ministry of Housing, Communities & Local Government. (2021). Supporting Families 2021-22 and Beyond.

193. Office of the First Minister and Deputy First Minister. (2013). Best Practice in Addressing Child poverty.

194. The Poverty Alliance. (2022). Child poverty Delivery Plan 2022-2026: Voices From Our Communities.

195. Joseph Rowntree Foundation. (2016). We Can Solve Poverty in the UK.

196. Joseph Rowntree Foundation. (2016). We Can Solve Poverty in the UK.

197. Ministry of Housing, Communities & Local Government. (2021). Supporting Families 2021-22 and Beyond.

198. CanChild.

199. Research in Practice. (2022). What is Early Help? Concepts, Policy, Directions, and Multi-agency Perspectives.

200. Molloy, D. (2019) Child poverty and Early Intervention.

201. Molloy, D. (2019) Child poverty and Early Intervention.

202. Ministry of Housing, Communities & Local Government. (2021). Supporting Families 2021-22 and Beyond.

203. Ministry of Housing, Communities & Local Government. (2021). Supporting Families 2021-22 and Beyond.

204. Greater Manchester Cost Benefit Analysis Tool.

205. Delivering Debt Advice during a Pandemic.

206. HACT. (2014). Measuring the Social Impact of Community Investment: A Guide to using the Wellbeing Valuation Approach.

207. Evaluation of Universal Infant Free School Meals.

208. The Real Cost of Christmas.

209. Fujiwara, D. (2010). The Department for Work and Pensions Social Cost Benefit Analysis framework.

210. Leck, C. (2018). A Social Return on Investment (SROI) Evaluation of the Houghton Project.

211. Big Lottery Fund. (2014). Wellbeing Programme: An introduction to Social Return on Investment.