Scottish Strategy for Autism: evaluation

2. Background and methodology:

In this chapter we set out the context and background to the ten-year Scottish Strategy for Autism and the approach for undertaking the evaluation.

Context

Autism (also referred to as autism spectrum disorder (ASD)) is the term applied to a range of neuro-developmental conditions which all affect how an individual communicates and relates to other people. Scottish Autism describes autism as, ‘a lifelong, developmental condition that affects the way a person communicates, interacts and processes information’. The National Autistic Society gives a similar definition, but adds, ‘disability’, ‘a lifelong developmental disability which affects how people communicate and interact with the world’. Other sectors of the autism community prefer a definition of difference over disability, stressing that autism lends a different way of perceiving and engaging with the world, but to describe it as a ‘condition’ or ‘disorder’ is incorrect and even offensive. However, disorder, condition and impairment are clinical terms and frequently appear in material written from this perspective.

The autism community is represented by many national and local charities and organisations, often set up or led by autistic individuals or their family members and all offer wide perspectives and contrasting insights into autism and its impact on individuals and families.

Research carried out in Scotland estimated that the national autism prevalence rate is 103.5 per 10,000 meaning that there are approximately 44,000 autistic people in Scotland. A better understanding and recognition of what it means to be autistic, and easier access to diagnosis, is leading to increasing prevalence.

Barriers and disadvantages faced by autistic people

There is consensus that autism exists on a spectrum and that it affects individuals differently; there can be no ‘one-size-fits-all’ definition. Autistic people face disadvantage in many areas of their life because of their neurological differences.

Although what it means to be autistic is becoming better understood, many professionals still have a lack of awareness and understanding of autism and the different ways in which it might affect people. The ‘invisibility’ of autism can also make it difficult for people to access services; with no obvious physical signs of disability, their way of interacting with people may be seen as simply ill-mannered, or alarming.

Receiving support can rely heavily on good social and communication skills when completing forms or taking part in assessments. This can affect people’s chances when taking exams or applying for a job, for example. At the same time, many talented, articulate individuals, adept at masking their differences, may not be given the support they need as professionals and employers may not recognise that they need help.

These challenges take many forms and can affect an individual’s physical and mental health, engagement in education, access to employment and services and participation in social and cultural activities.

Evolution of the strategy

The Scottish Strategy for Autism was the Scottish Government’s response to the inequalities experienced by autistic people and an acknowledgment that action needed to be taken to address them.

The Scottish Government and COSLA launched the strategy in 2011. The original document set out 26 recommendations and the vision that by 2021 'individuals on the autism spectrum are respected, accepted and valued by their communities and have confidence in services to treat them fairly so that they are able to have meaningful and satisfying lives’.

Following a report on the Foundation Phase of the strategy in 2014, the strategy was re-framed and refreshed in 2015 to an outcomes approach organised under four key areas:

• A Healthy Life: Autistic people enjoy the highest attainable standard of living, health and family life and have timely access to diagnostic assessment and integrated support services.
• Choice and Control: Autistic people are treated with dignity and respect and services are able to identify their needs and are responsive to meet those needs.
• Independence: Autistic people are able to live independently in the community with equal access to all aspects of society. Services have the capacity and awareness to ensure that people are met with recognition and understanding.
• Active Citizenship: Autistic people are able to participate in all aspects of community and society by successfully transitioning from school into meaningful educational or employment opportunities.

After further consultation with autistic people and their families, spokespeople from autism organisations and professionals working in autism services, and against a backdrop of evolving policy and increased devolution of powers, the strategy was re-launched in 2018. The four key areas outlined in 2015 continued as guiding themes for the final phase of the strategy (2018-2021).

Image with a timeline of the strategies reviews, developments and outcomes

• 2000: SG of services for people with disabilities led to some recommendations with focus on autism spectrum
• 2001: The public Health Institute Scotland Needs Assessment of Autistic Spectrum Disorders Report
• 2004: A two-year resaerch project resulted in the publication of The National Training Framework for Autism Spectrum Disorders
• 2006: HMIE conducted a review of provision for pupils on the spectrum with the National Autistic Society's Make School Make Sense campaign that led to the Autism
• 2007: SIGN guidance on assessment, diagnosis and intervention for individuals on the spectrum up to 18 years of age published
• 2008: Policy guidance for commissioners of services for adults on the spectrum was developed
• 2011: Scottish Strategy for Autism launched
• 2014: The Foundation Phase report detailed the progress mode during its first two years
• 2015: Autism strategy refreshed and reframed as an outcomes approach
• 2018: Engagement excercise led to an updated set of priorities, Scottish Strategy for Autism: outcomes and priorities 2018-21

National strategy, local delivery

In both its iterations, the national strategy presented a comprehensive overview of what actions were needed to address the inequalities experienced by autistic people. While it gave a broad vision and direction, actual implementation was expected to be undertaken at a local level.

To enable local areas to deliver on the national strategy, it was supported with funding and underpinned with advice and expertise. Initial investments took the form of a succession of funding rounds - the Autism Development Fund (2011-2015), Autism Innovation and Development Fund (2015-2018) and the Understanding Autism Fund (2019-2021).

From 2015 onwards, Inspiring Scotland managed the funding programmes and co-ordinated the project monitoring. Their role also included on-going support for the fund recipients with issues like grant management, fundraising and sustainability. Inspiring Scotland encouraged collaborative working between the projects by hosting events to bring the projects together and they offered pro-bono support from a network of professional volunteers including accountants, lawyers and marketing specialists. Inspiring Scotland also delivered more intensive capacity building support for projects from 2017-20 through the Scottish Government-funded Resilience programme. This included a leadership and coaching programme, a series of learning seminars and an online charity knowledge hub.

In addition to the support provided by Inspiring Scotland, Autism Network Scotland (ANS) and the National Autism Implementation Team (NAIT) helped to build capacity and expertise and act as a bridge between the national strategy and its implementation at local level, providing a hub of professional autism support and information. ANS worked with local authorities with a range of issues from consultation, through to implementation of their autism strategies. NAIT provided practitioner-to-practitioner support around education and inclusive practice, assessment and diagnosis and employment.

Governance of the Strategy

Within Scottish Government, the Autism and Learning Disability Team provided policy advice and were responsible for putting Scottish Government policy into practice. A governance group (the Autism Strategy Review Group) was set up to monitor progress against the strategy and this group provided strategic leadership and maintained an overview/governance of the strategy’s priorities. Membership included autistic people, representatives from the third sector, government, and professional experts.

Sub-groups were established as off-shoots of this main governance group where specific areas merited further focus, such as employment, diagnosis and consultation. These smaller groups were responsible for delivering guidance, direction and relevant expertise in the development and delivery of the priority work plans.

Alongside this main governance group, and sub-groups focussing on specific parts of the strategy, in 2017 a further group was established. The Cross-Party Group (CPG) on Autism acts as a channel of communication between the Scottish Parliament and autistic people, their families and carers, professionals and organisations. It is a forum to raise awareness and promotes the interests of autistic people, their families and carers; and it influences Scottish Government policy. In 2020, the CPG published The Accountability Gap Report, which was their review of the strategy.

Methodology

In the remainder of this chapter, we describe the approach that we employed to complete this evaluation. The diagram below summarises the methodology, and each stage is described in more detail overleaf.

Image showing the three phases of the evaluation

• Phase 1 - Planning & Preparation
  • Virtual inception meeting
  • Desk-based activity
  • Design of research tools
  • Confirmation participation stakeholders and local authorities
  • Scheduling interviews and workshops
• Phase 2 - Feildwork
  • Stakeholder interviews
  • Virtual workshops in local areas
  • Online survey
• Phase 3 - Analysis & reporting
  • Quarterly progress reports
  • Data analysis
  • Final report and executive summary
  • Visual minutes from the workshops

We describe the key elements of our methodology in detail below.

Desk-based activity

This stage involved reviewing key documents, including unpublished material, related to strategy implementation and delivery at a local and national level. We mapped the activity identified in the documents against the 26 recommendations from the original strategy, the strategy refresh and the outcomes and priorities identified for 2018-21. We shared our working document with key stakeholders in order for them to enhance the information and provide us with additional details and evidence.

This desk-based activity provided evidence of the activity delivered and progress achieved in implementation of the strategy, and helped us to identify topics for exploration in the interviews and workshops.

Stakeholder interviews

We interviewed 18 stakeholders from 14 organisations. These interviews explored stakeholders’ views of the progress made towards the recommendations and priorities laid out in the strategy, the impact of the activity delivered on autistic people and the services and charities that support them, and the lessons learned from the strategy. Appendix 1 contains a list of organisations represented by the interviewees.

As a result of COVID-19 restrictions, all interviews took place remotely via telephone, Microsoft Teams or Zoom.

Virtual workshops

Initially, we were contracted to faciliate six virtual workshops. These were selected through the desk-based research and in discussion with Autism Network Scotland (ANS), taking account of a balance of urban/rural areas, each area’s demographics, and ensuring a breadth of different health and social care partnerships and autism representatives across Scotland. Nine health and social care partnership areas were shortlised for the workshops, and ANS approached the autism leads in these areas to establish if they would be able to host a workshop. To expand the opportunity for more areas to be involved in the virtual workshops, the decision was taken by Scottish Government to offer a workshop to any area that requested one.

As a result, we facilitated 11 virtual workshops in:

• Aberdeenshire
• Dumfries & Galloway
• East Dunbartonshire
• Edinburgh
• Fife
• Glasgow
• Highland
• Midlothian
• North Lanarkshire
• Renfrewshire
• South Lanarkshire.

To recruit participants for the workshops, we liaised with the local autism leads and Inspiring Scotland and raised awareness amongst any individuals who contacted us about the evaluation. Participants included policy makers and service providers from the public and third sectors, autistic people and family members of autistic people. Table 2.1 shows the range of participants who attended the workshops.

Table 2.1: Types of participants

	Number of participants	Percentage of participants
Public sector	74	53%
Autistic people/family members of autistic people	42	30%
Third sector	24	17%

As a result of COVID-19 restrictions, all workshops were held online using either Microsoft Teams or Zoom. Both these packages allowed for the functionality of gallery views, screen sharing and breakout rooms which helped to maximise engagement in the virtual workshops and allowed smaller group discussions.

The workshops were led by a Director of Blake Stevenson and featured a pre-recorded message from the Scottish Government as well as a summary presentation from a local representative. Participants contributed to the discussion through breakout groups. The smaller breakout group discussions were facilitated by a member of our team from Blake Stevenson and a member of staff from ANS, who supported the workshops as independent facilitators. Participants were invited to discuss:

• how the national strategy had influenced local services for autistic people;
• the difference that local services had made for autistic people;
• how local services could improve; and
• what was needed at a local and national level for improvements to happen.

We also asked participants to take part in an online poll about the most pressing gaps in local autism services.

The sessions were visually-minuted by Jenny Capon, an artist and illustrator. These minutes (in Appendix 2) were sent to all workshop participants as a record of the discussions, and some of the images are also featured in this report.

Short online survey

In addition to conducting stakeholder interviews, we administered a short online survey of additional stakeholders to increase our reach and gain an understanding of their views and experiences. This survey was conducted using Snap Surveys and included three key questions:

• How have the priorities defined in the national strategy 2011-2021 helped to focus and resource effective autism services?
• How did the national strategy support and influence the delivery of accessible and relevant local autism services?
• How have locally delivered autism services impacted on the life of autistic people and their families?

The survey link was distributed by email and social media and we received 154 responses. We offered respondents the choice of completing the survey questions via telephone for those that preferred to do this, but nobody took up this option.

The online survey was deliberately short to ensure it was not too onerous for respondents and in recognition of the extensive consultation that had taken place over recent years for the strategy refresh and, more recently, the Accountability Gap report. However, the open-ended nature of the questions meant that respondents could give in-depth responses if they wished.

Table 2.2 indicates the different types of survey respondents, based on information provided by 151 respondents.

Table 2.2: Survey respondents ^[1]

	Number of participants	Percentage of participants
Autistic person	53	35%
Family/friend/carer	96	64%
Work/volunteer for a service	44	29%
Policy maker	13	9%

Additional submissions

Unusually, throughout the evaluation we were contacted separately by people who were keen to give their views about the strategy. We invited these people to attend a workshop if they lived in an area where a workshop was taking place, or, if that was not possible, we invited them to take part in the online survey. In nine cases, these individuals sent us additional information by email. These included detailed personal accounts of autism services and views of the strategy. We have reviewed these submissions and most of them are included in the analysis for this report and the remainder are detailed in Appendix 3.

Analysis and report writing

We analysed the extensive qualitative data we gathered from the interviews, workshops and survey in line with the evaluation questions. This involved a process of coding responses to allow us to identify key and recurring themes as well as any differences in viewpoints among different groups of participants.

The quantitative data collected through the online poll was collated and analysed using Microsoft Excel.

We held an internal team meeting to discuss the key emerging themes before holding a virtual discussion with the Scottish Government’s Autism and Learning Disability policy team to discuss the main themes that had emerged and to agree the structure of this report.

The results of our analysis are presented in this report.

Challenges and limitations of the approach

There were some challenges which we had to take account of in our methodology. The most significant of these was that the research took place during the height of the COVID-19 pandemic. This meant that all consultation had to take place remotely via web-based technology. Had the research taken place during “normal” times, face-to-face consultation would have been proposed for this type of research.

Online delivery of workshops may have deterred some autistic people who would have preferred a one-to-one approach from taking part. However, we took a variety of steps to make the workshops as welcoming and non-intimidating as possible by, for example, holding discussions in small breakout groups, providing clear guidance on how to engage during the sessions, e.g.use of the ‘hands up’ function in Zoom or Teams when anyone wanted to speak, and, where relevant, assigning support workers to groups with autistic people. We took advice from the local leads to ensure that participants were allocated to groups sensitively and appropriately. Those who did not feel comfortable taking part in a workshop were able to participate in the online survey.

Other issues arising included:

• High levels of interest from local authority areas to host a virtual workshop. All requests were accommodated, resulting in 11 workshops (five more than originally intended) being delivered.
• Ensuring that the recruitment process did not exclude anyone interested and able to take part. This was addressed by involving local leads in co-ordinating the workshop recruitment process.

As is frequently the case with qualitative research, the contributors to this report (who numbered 300+), are only a selection of people involved in shaping, delivering and accessing services designed for autistic people or their families and carers. Their views and experiences are therefore not necessarily representative of the wider autistic population, service providers or policy makers.