Social Security Charter measurement framework: co-design process

5. What is in the framework and what information will inform it?

The Framework

The Framework (http://www.gov.scot/ISBN/9781839602474) has four sections that match up with Our Charter. It includes a total of 77 measures. Some of the measures could be described as ‘objective’ as they relate to aspects of the system, for example, call waiting times, and others could be described as ‘subjective’ measures, like people’s feelings and perceptions.

The measures will need information or data to be collected to complete them; some will need quantitative data (numbers and percentages) and others will need qualitative data (narrative or words).

A table listing all the measures and the data that will be collected to inform them is set out in Annex C.

This section sets out the content of, and data requirements for, the Charter Measurement Framework.

• Section 1: A people’s service measures to what extent:

(i) clients experience a service that reflects the human rights values as set out in Our Charter and

(ii) Social Security Scotland staff are delivering the charter commitments

It consists of six columns containing 29 measures

• Section 2: Processes that work measures to what extent:

(i) processes work

(ii) services are accessible

(iii) places are accessible and convenient

(iv) the face to face assessment process for benefits for disabled people (and people with health conditions) minimises stress for clients

It contains four columns and 26 measures

• Section 3: A learning system measures to what extent:

(i) Social Security Scotland is a learning organisation

(ii) Social Security Scotland is involving clients

It has a total of four columns with 15 measures

• Section 4: A better future measures to what extent:

(i) the Scottish Government has effective processes of policy making

(ii) the Scottish Government promotes social security in a positive way

iii) devolved benefits are making a difference

It has a total of eight columns with eight measures.

Finally there is a summary page which will give an ‘at a glance’ overview of how Social Security Scotland is performing in each section, presented with a few important pieces of data.

Annex C sets out a full list of all the measures, what data will be used to inform them and when that data will be published.

Objective and subjective measures

All participants agreed that the Measurement Framework should have both measures that objectively record aspects of the system, and measures that reflect people’s feelings and perceptions (subjective measures). As SCoSS put it:

“Clearly, surveying the experiences of clients and staff, then following up with questions to find out more details, is exactly what's required to test if some of the Charter expectations are being met….(for) other Charter expectations, particularly relating to the more technical side of the system, …there may be other sources of information that would give a fuller picture”.

The final framework has more subjective than objective measures. This is because Core Group 2, in particular, strongly believed that the experience and feelings of clients are the most important indicator of the success of the system. For example, in a workshop where members ranked the proposed measures in order of importance, ‘% of clients who say they were treated with kindness’ came out on top, closely followed by ‘staff understand, and value, the human right to social security’.

The co-design partners also believed that people’s opinions and views provide a level of understanding that statistics and figures alone cannot achieve. One example of this is the issue of timeliness; while it may be useful to know how long it takes to answer enquiries, or process applications, this is different from finding out from clients whether they feel their time was wasted in their dealings with the agency. Things may be processed quickly, but if mistakes are made and clients have to follow up on claims unnecessarily, this amounts to a poor use of time.

Another example of why it is important to get information about people’s perceptions is when we want to know about particular processes, such as complaints. We could measure the number of complaints, and how many are responded to in a given time period, but this will not tell us whether clients were satisfied with the way in which complaints had been dealt with.

Qualitative and quantitative data

The measures in the framework need a mixture of quantitative numerical data and qualitative narrative data. A number of different collection methods can be used for each.

Data collection

Surveys

Many of the measures in the framework are based on perceptions of clients and Social Security Scotland staff; the main methods of information or data collection will be surveys, focus groups and interviews with social security clients, staff and management at Social Security Scotland and stakeholders.

A programme of client and staff insights research is already under way in the Agency. It includes snap surveys with clients following key interactions with Social Security Scotland, a Social Security Scotland staff survey, and the collection of workforce statistics, which includes diversity information. Further work planned includes an ‘all clients’ survey, as well as more research with staff on topics such as recruitment and training^[9]. The surveys will collect both qualitative and quantitative data, whereas focus groups and interviews will collect the qualitative data we need.

Management information

Other measures in the framework will use data from Social Security Scotland’s robust and expanding data collection systems. One example of this is call waiting times and application processing times, which will be used to measure the Charter commitment to handle applications and enquiries as quickly as possible. This type of data is called management information. You can find key data and information already published by Social Security Scotland on the Scottish Government’s website^[10].

Focus Groups and Interviews

Some of the measures in the framework call for more in-depth qualitative data; sometimes this will be collected through follow-up questions in surveys where respondents can write answers in their own words. In other cases, researchers will do additional work with certain groups; for example focus groups and interviews with clients who have experience of the assessment process. For some measures, case studies will be done so that lots of aspects of a process can be looked at at the same time. For the ‘you said: we did’ suggestion, for example, researchers would look at the types of comments and complaints made by clients, how they are recorded, how they are dealt with and by whom, and what the results are.

Other data collection methods

Some measures will need researchers to look through documents produced by Social Security Scotland and the Scottish Government and analyse them.

The framework also proposes an ‘accessibility checklist’ to be used for premises in which Social Security Scotland is based, which will be developed and implemented in consultation with clients.

There may also be data collected by academics or third sector organisations that can be used to inform the measures. Researchers will undertake work to explore any such data and use it where it is robust and relevant

A table matching the measures with data sources is included at Annex C.

Finally, the Social Security (Scotland) Act 2018 states that the Charter itself will be reviewed at least every 5 years and, as such, the Charter Measurement Framework will also be reviewed to reflect any changes made.

Changes will also be made, in consultation with clients, if the methods set out in the framework can be improved, or if new, better sources of data and evidence become available. Wherever possible, data from surveys and data collected by the agency will be analysed and reported by protected characteristics, such as ethnicity, geographical location, age, gender and so on. In this way, analysis will be able to show whether or not service delivery is experienced in the same way by different groups of people, and will enable any discrimination (intentional or otherwise) associated with particular characteristics to be addressed.