Child Poverty Action Group - advice, information and training services: evaluation

5. Conclusions and recommendations

5.1 Conclusions

Similar to the previous evaluation in 2011, CPAG in Scotland's second tier advice, information and training services have evaluated positively. This is clearly seen in both the secondary data provided to us by CPAG in Scotland, and in the primary data we collected through the survey questionnaire and stakeholder interviews.

CPAG in Scotland's services and staff were held in high regard

This was a common theme throughout the study and can be particularly clearly seen in the following:

• 100% of respondents to CPAG in Scotland's annual surveys between 2014 and 2019 reported that they were satisfied with the Advice Lines overall level of service and that it is the best place to get second-tier benefits advice. They also all reported that they felt that the staff had explained the advice clearly and were knowledgeable and approachable
• There has been a 42% increase in the use of the Advice Line in the last five years and this has been a steady increase, year by year
• The majority of the respondents to the Scottish Government survey felt that the Advice Line addressed the issues they had at hand and were satisfied with the advice they received (97%). A similarly high percentage reported that they felt the information provided in the CPAG in Scotland training improved their benefits-related knowledge and skills (96%) and that they felt more confident in their job role after completing the training (94%)
• All of the stakeholders from the in-depth interviews reported that they found the Advice Line to be particularly useful. The majority felt that the information resources on offer that they had used and were aware of were of high quality and they had no criticisms. With regards to training most of the stakeholders felt the quality was high and any criticism was in relation to geographical or financial barriers rather than structure or content
• The closing remarks of many of the stakeholders in the interviews were extremely positive of CPAG and the services they provide

The perceived continued need for CPAG in Scotland services in light of structural and societal changes (Universal Credit, Scottish Social Security and Brexit)

The survey and stakeholders from the interviews all made it very clear that the highest perceived challenges facing frontline advisers over the next five years will be around Universal Credit and Scottish social security, as well as the unknown economic and social effects of Brexit. Respondents commonly reported that they felt that demand for their services would continue to increase (90% of survey respondents and most of the interview stakeholders) and that they are starting to see it happening already. Stakeholders in the interviews made it particularly apparent that they felt the need for CPAG in Scotland's services would also steadily increase.

Assessing the extent to which CPAG in Scotland has delivered on the agreed outcomes set out in the Scottish Government grant letter

The evidence collected from primary and secondary sources indicates that CPAG in Scotland's services available to advisers and frontline support staff have led to an increase in benefit and tax credit knowledge and skills. As a result, these advisers and frontline support staff are able to provide accurate, high quality, effective advice and information to their clients.

Both the quantitative and qualitative research indicates that the services provided are considered essential by many of the frontline advisers

• 100% of respondents to CPAG's annual surveys between 2014 and 2019 that they felt that the Advice Line helps them achieve better outcomes for their clients and that they are better equipped to deal with similar cases after using it. Of these respondents the majority work with low income families as at least one of their client groups
• 94% of the Scottish Government survey respondents reported that the information provided through the advice line improved their benefits-related knowledge and skills, 96% that CPAG training improved their knowledge and skills, and 95% that CPAG information resources did as well
• Stakeholders from the interviews reported, fairly consistently, that the services provided by CPAG are of very high quality, help them significantly when they have issues, and are otherwise essential for them to be able to perform their duties in their respective positions

The review of the primary and secondary data has shown that, on the whole, CPAG services have contributed to advisers' ability to successfully achieve income maximisation for their clients.

• All of the CPAG Advice Line survey respondents between 2014 and 2019 reported that the Advice Line helped them achieve better outcomes for their clients
• The majority of respondents to the Scottish Government survey reported that they felt that the information resources (such as the Welfare Benefits and Tax Credits Handbook) (93%), the online resources (such as fact sheets and e-bulletins) (91%) and the Advice Line (91%) have contributed to income maximisation for their clients. Slightly fewer reported that the training and website contributed (88% and 83% respectively) and, while these percentages are still high, these are areas which could perhaps be explored for some improvement

5.2 Recommendations

The following recommendations are offered for consideration. Some of the recommendations made in the previous 2011 evaluation have not been actioned. These have been reflected upon and included when relevant.

1. The qualitative research found, similar to the 2011 evaluation, that many stakeholders do record the impact on their clients regarding income maximisation. This is generally done electronically and in a number of different systems. It was previously suggested that there may be scope to work with stakeholders to assess the impact on individual clients' income maximisation and/or the degree to which the organisation contributes to the impact of frontline agencies in this regard. However, probing further in this line of questioning in our research we found a number of barriers. Many stakeholders reported that it would either be impossible or incredibly difficult to draw this type of data from their systems while others felt that the information might not be recorded as often or as accurately as to be valuable. It might still be helpful for CPAG in Scotland to establish an advisory group to discuss further options with regards to developing a simple form or software of some kind that could be shared with stakeholders for easily recordable and accessible income maximisation data.

2. The e-learning platform has been expanded and updated since the recommendation for it to be in the 2011 evaluation, however there is certainly scope for it to be expanded further. There was clear interest amongst stakeholders in the interviews and some helpful suggestions for what to cover. In-person training is still inaccessible (due to geographical and financial barriers) for many advisers across Scotland, and development of key trainings that strike the balance between educating new staff and increasing the impact and efficiency of those that have been in post for a number of years would prove to be immensely useful.

3. Whilst the majority of feedback about the Advice Line was positive there is certainly a perceived need and strong desire for increased capacity. Responses to both the quantitative and qualitative work around this suggested that the most common complaint with regards to the Advice Line was not being able to get through and having to wait. With a marked increase in Advice Line usage over the last five years and a perceived increase of need for advice services in the near future surrounding the rollout of Universal Credit and Scottish social security, and the uncertainty surrounding Brexit, it would be helpful to increase the number of advisers and extend the opening hours throughout the week. This could be done by CPAG in Scotland with an increased budget from Scottish Government or effective reallocation of resources currently available.

4. Online versions of the major handbooks, particularly the Welfare Benefits and Tax Credits handbook, appear be of particular interest to some of the stakeholders. Those that work remotely or travel a lot may find it helpful to be able to access the information on a work iPad or laptop rather than needing to carry the handbook(s) with them. It was made clear by numerous stakeholders in the interviews, however, that if the handbooks were to be upgraded into a more interactive online format, it would still be preferential to keep the paper versions in print as well. It is therefore recommended that, should CPAG in Scotland focus on providing resources primarily in digital format, paper versions of the major handbooks should still be available for purchase by those that want or need them.

5. It was clear from the stakeholder interviews that there is significant interest in CPAG in Scotland providing template letters and/or digitally generated case letters. All of the stakeholders felt it would be useful and many were quite vocal about just how useful they felt it would be. A key point was made multiple times, however, that it would be best if they could be made so that they could be edited and tailored to each individual case by the frontline advisers.

6. During the qualitative interviews it became apparent that some advisers were not aware of the full range of services available from CPAG in Scotland. These same respondents reported that they would keen to use these services had they known that they existed. It is therefore recommended that CPAG in Scotland examines how it may more effectively publicise the range of services on offer to frontline advice organisations and individuals.

7. With stakeholders reporting that CPAG in Scotland's welfare rights services are essential and remote working now the norm for the time being due to the Coronavirus pandemic, a focus on providing resources digitally, training online and ensuring Advice Line capacity is sufficient is a clear priority.