Disability and Carers Benefits Expert Advisory Group: review

5. Facilitators and inhibitors to the Group

Answers to the open questions in the survey questionnaire shed light on what the Group members thought were the strengths of the Group that enabled it to be effective and make an impact. Below are the most frequently mentioned themes that emerged spontaneously in answers to an open question in a survey questionnaire which asked directly about enabling factors:

• Members' commitment
• Fit-for-purpose membership
• Effective chair and deputy chair
• Good relationships and communication with the Scottish Government

On the flip side, the following themes appeared most frequently as factors that may have inhibited the Group's ability to be more effective and fit for purpose:

• Constraints of time and capacity issues
• Resources of the secretariat
• Intra-group cooperation
• Attendance issues

The discussion to follow unpacks these and other themes to draw a more nuanced picture of the functioning of the Group to date.

5.1. Members' commitment

Most respondents agreed (nine respondents) or strongly agreed (six respondents) that Group members had been engaged and committed to the Group's affairs. Only one respondent was unsure about this and none disagreed. In addition, commitment and contribution by team members was one of the most frequently mentioned themes in a question about the enablers of the Group's achievements to date.

These views are reflected in the data on attendance to general meetings. The Group had eight general meetings between April 2017 and November 2018. The average attendance on meetings was fourteen members, with the highest attendance on a single meeting being seventeen and a lowest attendance of ten. Three members attended all meetings; only four members attended half or less of the meetings and one of these four members did not attend any meetings at all. On average, a single member attended six meetings. In line with this, survey responses indicated that non-attendance was limited to a small handful of members. Nevertheless, there were views that this problem had a negative impact on the meetings and that it should be addressed.

In addition, some respondents complained that substitutes were often not sent by members who couldn't attend. However, DACBEAG's Terms of Reference clearly state that 'meetings will be attended by named members, and support workers if required, only' and that substitutes will usually not be accepted, apart from by prior agreement with the Chair'. In line with this, one Group member argued also that sending substitutes is not an appropriate solution to absenteeism because members of the Group should attend meetings as individuals with their own views, experiences, and skills, not as representatives of organisations who can be substituted by other employees from these organisations. Moreover, it was noted that the strength of the Group was contingent on the quality of relationships among Group members and with officials and ministers, and that continuity was essential for these relationships. Because many of the discussions in meetings drew upon discussions the Group had previously, sending substitutes was seen as ineffective.

In addition to the problem of attendance and participation in meetings, a point was made that the workload of the Group was distributed unequally among members due to a particular focus of workstreams:

'The Group is large and diverse (which is good) however there has been an aspect of uneven distribution of workload due to the purpose being centred on three or four areas of heavy workstream activity which has fallen to about one third of the Group and their representatives.'

The perception of commitment of Group members in general (as discussed above) broadly corresponded to self-perceived commitment - namely, nine respondents agreed that they had been active participants in the Group's work and further four participants strongly agreed with this statement; three participants were unsure and none disagreed. Furthermore, all but one respondent said that they spent time working for the Group's work programme outside of the meetings.

5.2. Fit-for-purpose membership

Respect for the expertise, knowledge and experience of fellow members was apparent in responses to open questions in a survey questionnaire and during interviews. Indeed, membership was seen as one of the key strengths of the Group. In line with this, there were eleven respondents who agreed with the statement that membership was fit for purpose; two were unsure and two others disagreed with this opinion. Qualitative data suggests that critical views on membership are likely to be related to the issue of low attendance by some members - a problem that was limited to a handful of individuals (see previous chapter).

However, it also surfaced in the interviews that the Group may be short of expertise in some areas. Examples given by the interviewees were related to expertise in local delivery of benefits and to skills needed to provide detailed technical advice as well as operational advice. There was also a view that it was important to regularly refresh the membership to make sure that the Group had the right skills and that members themselves should self-reflect whether their own skills were relevant for the Group.

Survey data sheds some light on the latter point (i.e. the self-perceived alignment between members and the Group). First of all, most respondents felt that the Group provided a good platform for them to make a contribution to the Group's purpose - nine respondents agreed and four strongly agreed with such a statement; three respondents were unsure but none disagreed. Some responses to open questions in the survey questionnaire indicate that there may have been a suboptimal alignment between professional interests/expertise of members and the focus of Group discussions.

Overall, ten respondents agreed and one strongly agreed that being a member was in line with their expectations whereas three disagreed and two were unsure about this. Notably, the number of people who disagreed and were unsure about this statement was somewhat higher than in responses to most other questions in the survey whereas the number of people who strongly agreed was lower. Thus, these responses may be seen as indicating a possible mismatch between expectations about membership and what it actually involves.

5.3. Effective chair and deputy chair

The effectiveness of the chair and deputy chair were another theme that frequently and spontaneously surfaced in responses to open questions in a survey questionnaire and in interviews with members of the Group. In line with this, five respondents strongly agreed and ten others agreed with a statement that the general meetings were conducted in a way that enabled the Group to fulfil its remit. Only one respondent was unsure about this.

There was a view expressed by one of the members that although the Group chair strived to ensure that the voices of members were equally valued, there may have been occasions when certain views within the Group were curtailed. However, there is no indication in the data that this may be a more widely held view.

5.4. Good relationships with the Scottish Government

Overall, the relationship with Ministers, and the Scottish Government more broadly, was the third most frequent theme that emerged spontaneously in responses to the open question which asked respondents to identify what enabled the Group to be effective. This relationship was described with the following adjectives: 'good', 'effective', 'constructive', and 'honest'. In addition, support by the secretariat was frequently praised. However, there were also some critical views.

Before these are discussed, it is worth noting that this review is just a snapshot of the Group and its relationships with stakeholders. There has been a trajectory of development within the Group and in its relationship with officials. Both the Group members and the officials reported that many of the challenges that they highlighted had already been resolved or that they occurred in the early stages just after the Group was set up. For example, one member of DACBEAG noted that engagement with officials became closer and more consistent from the start of 2018 following a letter of clarification from the then Minister for Social Security. Views of officials seemed to corroborate this opinion. Several officials noted that more recent advice was more impactful than the early advice produced by the Group and explained that this was in large part thanks to a closer collaboration between officials and the Group (e.g. more frequent meetings and phone calls, presentations by officials during the Group's meetings).

5.4.1. Clarity of steer from the Scottish Government

Some interviewees felt that the steer from the Government was not clear enough. However, as discussed in chapter 4.2 of this report, this critical point needs to be clarified. First of all, not all interviewees would agree with this view. Secondly, officials argued that the Group was not given a highly detailed steer intentionally, even when requests to look at specific problems were made. This may have resulted in the perception among some members that the steer was at times not clear enough. Lastly, government officials said that they were not aware that the Group had found some of the steer unclear as no such feedback had been received from the Group. There was an invitation to provide such critical feedback and assurance that the Government can provide a more specific steer in the future if this is what the Group needs.

5.4.2. Information sharing

The second critical view on the relationship between the Group and the officials was related to information sharing. There were views in the Group that the Government shared information with the Group late. On the flipside, there was a recognition that the rapid pace of developments in social security reforms may limit the ability of officials to provide information more timeously. There was also an invitation made by one official for the Group to ask for information more proactively.

5.4.3. Clarity on involvement and contribution

Interview data indicated that there may have been occasions when clarity on the Group's contribution to policy making was lacking when the Group sought to provide advice proactively. In the absence of a specific request by the Government, there was an expectation among officials that the Group would be clear about the sort of contribution it sought to make. However, some officials reported that at times there was no such clarity.

Even though all pieces of proactive advice were seen as 'extremely helpful' by the officials, the above suggests that the Group could achieve further impact by developing a more concrete involvement with officials in areas where it proactively sought to give advice.

At the same time, officials may also want to develop better ways of utilising the Group's expertise in cases where the Group seeks to provide advice proactively. A quote below illustrates how this may have been stalled by uncertainty about how to engage with the Group:

'But in fairness, maybe that's as much on us as it is on them. Maybe we should have been more proactive in going along with the ask. So rather than encouraging the discussion, maybe we should have kind of said 'well, here's the task'. But then I'm not sure I'm empowered to do that because they have a remit and agenda so I don't know how that fits.'

The above corroborates a view of one Group member who pointed out that the Group had interactions with different individuals within the Scottish Government who had a differing levels of understanding regarding the role and remit of the Group.

5.5. Time constraints and capacity issues

Some officials noted that the ability of the Group to produce advice quickly enough was one of the concerns about the effectiveness of the Group. It was suggested that the Group needs to become more effective at delivering work to tight timelines. From the Group's perspective, half of survey questionnaire respondents (i.e. eight members) agreed that timescales for recommendations and advice set by the Scottish Government had been reasonable but none agreed strongly. A further five respondents were unsure about this and 3 disagreed. When thinking about workstreams specifically, six respondents (out of eleven) agreed and one strongly agreed that the timescales for recommendations and advice set by the Scottish Government were reasonable; three respondents were unsure and one disagreed.

On balance, information in Table 1 (p. 8) suggests that it took the Group approximately three months on average to produce advice from inception. For some of the more complex or contentious advice notes, this may reflect the quarterly nature of Group meetings - for example, it has been noted that the Group was reluctant to sign off such notes via email in between meetings, although they did that by necessity on various issues. It is for the key stakeholders to discuss and agree whether such timescales and speed of work are sufficient or not, and what could be done to make improvements. To assist this discussion, the subsections below identify the key capacity issues that may inhibit the Group from making improvements in its ability to work to tight timelines.

5.5.1. Members' capacity

Six respondents agreed and three strongly agreed that the scale of demands on their time had been reasonable. However, as many as five respondents were unsure about this and two disagreed. This represents one of the highest 'unsure' and 'disagree' response rates among all questions asked in the survey questionnaire.

Moreover, in response to open questions in the survey questionnaire, many members noted that members' time has been a major constraint on their ability to contribute more to the Group. Their involvement in the Group was seen by some members as a volunteering activity that they could pursue only to the extent to which their day jobs allowed. Below are just a few illustrative quotes from different respondents that highlight this problem:

'It's a volunteer group and everyone is busy with their day jobs'

'Time, I think, continues to be an issue both for individual group members and for the group as a whole.'

'I am acutely aware that I have not been able to contribute to the group in the way I would have wished due to time limitation.'

'Time is the only factor!'

Interview data indicated that capacity to devote time to the Group may be particularly limited for members from small organisations as well as members from the academic sector. Larger organisations have at times been able to pull in additional resources to conduct specific work by delegating work to other employees that were not members of the group. However, organisations with a handful of employees may not have this capacity. Likewise, it was suggested by another interviewee that members from the academic sector, who tend to have a highly specific expertise, may not be able to pull in additional support from within their organisations.

5.5.2. Group's capacity

Overall, general meetings of the Group were seen as effective by most members: six out of sixteen respondents to the survey questionnaire strongly agreed and seven agreed that the general meetings were productive; three respondents were unsure and none disagreed.

However, responses to open questions in a survey questionnaire and in interviews indicated that meeting agendas were usually packed and could not be fully covered during some meetings. It was common for meetings to run over the scheduled time. It is not sensible to attribute these problems to the way the meetings were chaired because several interviewees pointed out that the number of issues that the Group had to discuss was disproportionately large relative to the frequency of the meetings and that there were pertinent voices raised during meetings which could not have been silenced simply because of time constraints.

As a solution, several members suggested extending the duration of meetings. An alternative solution proposed was to increase frequency of meetings or to organise ad-hoc meetings when necessary to help complete specific pieces of work. However, it was noted that these solutions would likely result in further reduction of attendance as even the very committed individuals find it challenging to combine their day jobs with attending the meetings. Here we can see how the capacity issues discussed in the previous section have an impact on the capacity of the Group as a whole. On balance, three respondents strongly agreed and eight agreed that the frequency of the general meetings was adequate, three respondents were unsure and two disagreed.

There was also a view that time constraints faced by individuals limited the capacity of the Group to produce the kind of evidence-based advice that the Government may have expected. More specifically, it was suggested by more than one member that the Group may not be able to produce evidence that is based on a targeted and systematic analysis of evidence or literature because of capacity issues. Because of these issues, it was argued, the advice would necessarily need to be grounded primarily in members' expertise from their professional life. It was argued that there are obvious limits to what the Group can do as a non-statutory body and that it was for the Ministers to decide whether this model was good enough or whether it should be augmented:

'This won't be in a sense of what [a] literature review says; it will be evidence based on people's expertise from their professional life. It's for Ministers to decide whether this model is good enough or whether it should be augmented.'

Although examples of work where the Group was very effective in producing relevant advice within short period of time were given, these were qualified by a range of conditions (e.g. the nature of an issue being specific, high commitment of time by group members) that may not always be present. This raises questions over the sustainability of such effective performance.

There was also a view within the Government that the size of the Group may have been another factor negatively impacting on its ability to deliver. Relatively large membership was perceived by some officials as posing difficulties for organising meetings, for attendance at those meetings, and for gathering views from members. In contrast to this, there was a view within the Group that the Group being relatively large is an asset in terms of providing it with expertise necessary to work within its remit.

5.5.3. Secretariat's capacity

On the one hand, all respondents felt that the secretariat took forward decisions of the Group effectively. There was an equal spilt between those who agreed and those who strongly agreed with this statement (i.e. eight responses for each category). In addition, a number of positive comments about the secretariat emerged spontaneously in answers to open questions about the strengths of the Group.

On the other hand, several members felt that the secretariat remained under-resourced even after the arrival of the Modern Apprentice. It was suggested that the commitment and professionalism of the existing secretariat could not fully make up for the fact that there were still only '1.5 people' working in it (i.e. one person on a full-time and one on a part-time basis). Arguments have been raised that a better resourced secretariat would allow the Group to produce advice more quickly. For example, it was suggested that a better resourced secretariat, in particular one with additional support from senior-level managerial and analytical staff, would increase the analytical and stakeholder engagement capacity of the Group. Another interviewee argued that a more resourced secretariat could pull together group discussions and turn them into advice much quicker.

Contrary to this, there was a view within the Government that the secretariat did produce pieces of advice quickly but they were then not processed adequately fast by the Group. It was also suggested that sign-off procedures may have delayed advice progression - this issue is discussed in chapter 5.7 on workstreams. Furthermore, any delays should be seen in the context of the members' and Group's capacity issues discussed in the two previous chapters. There was also a view in the Government that the secretariat had experienced capacity issues but mostly in the early stages before the recruitment of the Modern Apprentice. It was argued that these issues had improved and resources currently available were not seen as particularly inhibiting. Moreover, it was noted that offers of resource have been made but the Group was yet to put forth a definitive request.

5.5.4. Information sharing and capacity issues

As discussed in chapter 5.4.2, some members of the Group believed that the Scottish Government shared information with the Group late. This may have had a negative impact on the Group's ability to mobilise its resources to develop detailed, challenging and timely advice. At the same time, there was an indication that members of the Group recognised the pressures under which officials operated and that these may have made it difficult for them to share information more timeously.

Interview data indicated that keeping the Group sighted of broader and longer term plans as opposed to ad-hoc requests for advice on specific issues/benefits may help the Group better prioritise and manage its workload with its limited resources. In particular, there was a view that the Group has largely focused on individual benefits issues in isolation from the larger picture of social security reform. It was suggested that it may be more effective to focus on more generic, cross-cutting themes. It was explained that this required officials to keep the Group sighted on longer term plans. In line with this, it was suggested that the Group could benefit from having a wider sense of a range of ongoing and planned actions in the Government in order to be able to strategically select issues of fundamental importance and those where it could really add value instead of 'getting bogged down in small-scale issues'.

A view in the Government was that there was exchange of this sort of information. Officials explained that they did keep the group apprised of the overall policy landscape and trajectory of benefits delivery and that they shared all relevant information that they were allowed to share. It was also noted by several interviewees that collaboration between the Group and officials became closer over time. For example, one interviewee explained that the early-stage approach was to give an initial steer to the Group with some supporting information and then not hearing from the Group for a long time until the Group came with some advice. At that point, it was argued, officials were not aware of the discussions and thought process which informed this advice, nor what was the evidence that supported it. Over time, however, collaboration has become closer and more effective (i.e. more frequent invitations for officials to give presentations to the Group, more frequent phone calls).

5.6. Intra-group collaboration

All respondents agreed that the Group works harmoniously. Among them, exactly half agreed with this statement and the other half strongly agreed. However, qualitative survey and interview data indicated that several members felt that they did not know each other well and that this may have inhibited cooperation and a fuller utilisation of expertise within the Group. It was also argued that this may have impacted on the confidence of the Group to produce more challenging advice.

It was suggested by several interviewees that the Group could have benefited from team building early on. Although the Group did have an Away Day, this was not seen as particularly helpful in terms of members getting to know each other better.

5.7. Workstreams

Most respondents (eleven respondents) said that they had experience of working in one of the workstreams within the Group; five respondents said that they had not had such experience. In addition, seven respondents agreed and two respondents strongly agreed that workstream(s) they worked in were able to produce in-depth and detailed analysis and recommendations. One respondent was unsure and one strongly disagreed. In addition, collected data suggests that several workstreams included non-members who were invited by the Group for their particular expertise and contributed to drafting advice.

There were many positive references to workstreams in answers to open questions. These were related to the commitment to workstreams shown by members as well as learning opportunities that workstreams presented to those involved.

However, it was also noted that effectiveness of some of the workstreams was limited by a lack of timely, clear and sufficient information. Some respondents argued that the consequence of this was that some workstreams were delayed in starting work in earnest and the momentum was lost.

Moreover, it was also suggested that the Group could develop a better procedure for signing off workstream outputs. Related to this, another member suggested that the problem was not simply about how workstream outputs were signed off but more fundamentally about the relationship between workstreams and the rest of the Group. In particular, it was argued that workstream output should not just be reported to the Group for signing off but instead the Group should engage more actively with workstream outputs by feeding into them more extensively.